-- Logs begin at Wed 2022-07-13 19:56:53 UTC, end at Wed 2022-07-13 21:06:14 UTC. -- Jul 13 20:27:08.088647 dv-cinder-849690-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 13 20:27:08.102635 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 13 20:27:08 2022] *** Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: nodename: dv-cinder-849690-1-iscsi Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: machine: x86_64 Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: clock source: unix Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: pcre jit disabled Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: detected number of CPU cores: 4 Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: current working directory: / Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: detected binary path: /usr/bin/uwsgi-core Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: your processes number limit is 47680 Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: your memory page size is 4096 bytes Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: detected max file descriptor number: 2048 Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: lock engine: pthread robust mutexes Jul 13 20:27:08.103180 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: thunder lock: enabled Jul 13 20:27:08.103911 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 13 20:27:08.103911 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 13 20:27:08.146306 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: Python main interpreter initialized at 0x55f17fb5d9c0 Jul 13 20:27:08.146306 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: python threads support enabled Jul 13 20:27:08.146306 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: your server socket listen backlog is limited to 100 connections Jul 13 20:27:08.146306 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: your mercy for graceful operations on workers is 90 seconds Jul 13 20:27:08.146670 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: mapped 403077 bytes (393 KB) for 2 cores Jul 13 20:27:08.146670 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: *** Operational MODE: preforking *** Jul 13 20:27:08.146985 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: *** uWSGI is running in multiple interpreter mode *** Jul 13 20:27:08.146985 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: spawned uWSGI master process (pid: 80405) Jul 13 20:27:08.147222 dv-cinder-849690-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 13 20:27:08.151573 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: spawned uWSGI worker 1 (pid: 80415, cores: 1) Jul 13 20:27:08.151573 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: spawned uWSGI worker 2 (pid: 80416, cores: 1) Jul 13 20:27:08.151573 dv-cinder-849690-1-iscsi devstack@keystone.service[80405]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 13 20:27:13.482364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 20:27:13.482364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: from cryptography.utils import int_from_bytes Jul 13 20:27:13.483160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 20:27:13.483160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: from cryptography.utils import int_from_bytes Jul 13 20:27:13.489900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 20:27:13.489900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: from cryptography.utils import int_from_bytes Jul 13 20:27:13.490403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 20:27:13.490403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: from cryptography.utils import int_from_bytes Jul 13 20:27:13.530875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 20:27:13.531064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 20:27:13.531220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] command line args: [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 20:27:13.531529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 20:27:13.531699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ================================================================================ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 20:27:13.531988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] admin_token = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.532232 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config_dir = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.532389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.532600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config_source = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.532733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] debug = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.532957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.534328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.534328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.534328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.534328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.534328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.534328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_config_append = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.534701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.534701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_dir = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.534811 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.535023 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.535267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.535535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.535772 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.535950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.536117 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.536341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.536477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 20:27:13.536575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.536656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 20:27:13.536840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.537064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] command line args: [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 20:27:13.537238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.537404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 20:27:13.537583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.537693 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ================================================================================ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 20:27:13.537922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.538085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] admin_token = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.538237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.538404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config_dir = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.538582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.538748 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.538748 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config_source = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.539010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.539180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] debug = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.539180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.539545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.539685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.539841 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] publish_errors = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.539957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.540095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.540206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.540327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.540464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.540577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.540699 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.540811 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.540921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_config_append = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.541037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.541171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.541171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_dir = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.541387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] standard_threads = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.541556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.541679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.541791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.541907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.542026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.542145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.542145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_journal = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.542364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.542502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_json = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.542616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.542738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_stderr = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.542857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.542992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_syslog = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.543105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.543216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.543216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.543436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.543565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.543687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.543819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.543932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.544050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.544161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.544277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.544390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.544500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.544680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.544796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.544914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.545025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.545171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.545312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.545431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.545547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.545547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.545547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.545865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.545865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] publish_errors = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.546383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.546495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.546613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.546732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.546856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.546970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.547086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.547205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.547316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.547433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.547591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.547713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] standard_threads = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.547831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.547952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.548064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.548175 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.548296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.548409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.548409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_journal = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.548629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.548741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_json = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.548851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.548971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_stderr = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.548971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_syslog = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.549196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.549308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:27:13.549418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.549533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.549655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.549655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.549865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.549976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.550087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.550198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.550389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.550503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.550614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.550725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.550836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.550946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.551065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.551179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.551289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.551399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.551548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.551660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.551789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.551903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.552014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.552125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.552236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.552352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.552476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.552595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.552708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.552829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.552940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.553050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.553169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.553282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.553393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.553509 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.553509 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.553729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.553842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.553953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.554063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.554174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.554291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.554410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.554523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.554634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.554749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.554861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.554971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.555090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.555204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.555321 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.555439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.555594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.555706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.555894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.556017 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.556130 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.556240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.556352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.556462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.556580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.556693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.556804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.556916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.557034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.557146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.557264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.557379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.557490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.557601 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.557601 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.557815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.557918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.558888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.559046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.559205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.559322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.559434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.559668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.559786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.559898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.560023 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.560128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.560238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.560356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.560460 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.560571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.560690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.560801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.560801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.561024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.561128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.561238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.561350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.561468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.561591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.561784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.561909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.562013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.562123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.562242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.562355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.562466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.562578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.562689 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.562800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.562919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.563032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.563143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.563254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.563370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.563524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.563653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.563768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.563880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.563990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.564101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.564212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.564330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.564452 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.564565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.564677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.564788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.564788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.565004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.565117 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.565228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.565338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.565449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.565560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.565679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.565794 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.565905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.566015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.566117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.566226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.566346 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.566459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.566571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.566681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.566800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.566920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.567107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.567228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.567342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.567466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.567581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.567702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.external = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.567823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.567947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.568075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.568189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.568302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.568414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.568525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.568643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.password = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.568758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.568869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.token = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.568992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.569106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.569218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.569336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.569336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.569553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.569666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.569777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.569895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.570017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.570131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.570242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.570353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.570464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.570575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.570694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.570807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.570920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.571032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.571143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.571253 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.571394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.571545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.571673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.571788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.571908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.572022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.572135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.572248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.572368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.572490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.572604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.572716 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.572829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.572941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.573065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.573065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.573281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.573281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.573491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.573603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.573725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.573840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.573952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.574070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.574185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.external = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.574297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.574724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.574842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.574955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.575067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.575181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.575293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.575416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.password = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.575561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.575675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.token = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.575787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.575915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.576030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.576154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.576269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.576388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.576502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.576620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.576733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.576855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.576970 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.577083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.577194 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.577307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.577418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.577542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.577662 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.577775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.577886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.578004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.578118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.578118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.578367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.578501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.578620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.578737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.578850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.578973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.579093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.579207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.579319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.579438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.579600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.579731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.579848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.579960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.580073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.580185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.580297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.580417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.580533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.580662 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.580777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.580889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.581001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.581192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.581309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.581421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.581541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.581654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.581776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.581891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.582003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.582114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.582227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.582338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.582458 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.582573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.582685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.582797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.582909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.583021 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.583021 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.583240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.583353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.583483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.583599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.583722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.583847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.583962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.584074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.584186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.584297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.584409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.584528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.584650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.584763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.584874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.584985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.585096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.585215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.585329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.585441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.585553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.585666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.585785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.585907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.586027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.586140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.586251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.586363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.586474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.586594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.586710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.586821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.586933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.587045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.587156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.587276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.587390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.587549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.587663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.587776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.587888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.588012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.588127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.588239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.588351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.588467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.588467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.588685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.588799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.588912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.589024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.589136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.589248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.589368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.589481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.589593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.589705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.589817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.589929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.590049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.590163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.590275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.590387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.590506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.590619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.590740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.590854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.590967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.591079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.591191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.591311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.591426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.591554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.591678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.591792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.591904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.592029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.592145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.592257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.592369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.592480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.592592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.592712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.592826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.592968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.593084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.593197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.593309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.593429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.593544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.593657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.593769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.593881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.593994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.594115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.594230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.594341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.594454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.594566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.594678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.594799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.594913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.595025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.595138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.595250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.595363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.595518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.595636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.595749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.595862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.595987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.596100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.596222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.596336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.596448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.596560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.596672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.596784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.596904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.597019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.597131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.597243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.597355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.597355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.597579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.597694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.597806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.597918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.598920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.599139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.599139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.599351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.599481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.599608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.599723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.599835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.599947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.600059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.600171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.600291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.600406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.600518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.600630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.600742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.600854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.600974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.601088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.601200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.601312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.601425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.601537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.601657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.601772 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.601884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.601996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.602108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.602219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.602339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.602454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.602566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.602678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.602790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.602902 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.603089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.603205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.603318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.603318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.603545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.603670 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.603785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.603897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.604009 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.604122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.604234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.604355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.604469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.604582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.604694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.604806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.604917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.605038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.605152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.605265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.605377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.605489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.605602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.605602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.605822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.605934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.606051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.606164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.606276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.606396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.606510 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.606622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.606736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.606847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.606959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.607079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.607193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.607304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.607417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.607560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.607673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.607798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.607913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.608025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.608137 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.608249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.608364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.608484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.608598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.608710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.608823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.608934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.609046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.609176 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.609291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.609404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.609516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.609628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.609740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.609861 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.609976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.610088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.610200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.610312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.610424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.610544 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.610658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.610770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.610770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.610987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.611099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.611220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.611334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.611446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.611608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.611721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.611845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.611971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.612097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.612212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.612323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.612436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.612548 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.612660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.612780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.612895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.613008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.613120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.613232 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.613360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.613481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.613595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.613707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.613819 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.613938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.614053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.614174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.614289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.614401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.614401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.614611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.614723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.614843 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.614958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.615071 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.615183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.615296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.615408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.615553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.615675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.615790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.615902 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 20:27:13.616248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.616248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.617150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.617150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.617150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.617150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.617150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.617150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.617150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.617150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.618005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.618005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.618005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.618005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.618421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.618421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.618660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.618866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.619111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.619358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.619620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.619870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.620107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.620353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.620586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.620828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.621065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:27:13.621235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 20:27:13.642951 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.643181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.643405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.646807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.647042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.647270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.651638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.655656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.679168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.679389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.681302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.681511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.821091 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.821762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.869631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.869870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.869970 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.870231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.934167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.934429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.934575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.934660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.934801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.934883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.934946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.935068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.935140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.935273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.935362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.935524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.960637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.960812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.960946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.961850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.962013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.972839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.973058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.985566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.985776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.985988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.986136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.986217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.986280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.986402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.986480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.986627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.986709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.986839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.986920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.987053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:13.987297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:27:14.004063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.004269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 20:27:14.004392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.004630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 20:27:14.004793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.005019 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 20:27:14.005144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.005328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.005457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 20:27:14.005600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.005796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.005888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.006013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.006096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.006225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.006374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.016855 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.017127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.017252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.017406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.017535 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.017676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.017819 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.018074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.025810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.026302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.029080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.029352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.029613 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.029740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.029888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.030036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.030131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.030256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.030382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.030605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.030690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.030820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.030906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.031029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.031111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.031232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.031368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.031544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.031671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.031799 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.031996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.032282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.058065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.058735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.059695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.059939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.062648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.062648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.065241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.065512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.065787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.066151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.066436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.066715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.073208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.073432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.074155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.074731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.078256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.078537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.078815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.079070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.079348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.079512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.079600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.079674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.079890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.080017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.080119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.080199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.080452 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.080725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.080996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.081261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.098567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.098834 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.099728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.100004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.103561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.104519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.107070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.107357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.108161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.108455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.112509 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.113664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.116100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.117295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.120024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.120308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.120585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.120868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.120994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.121154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.121239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.121486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.121776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.122060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.122341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.143731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.144005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.144128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.144263 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.144369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.144486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.144685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.144887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.152506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.152867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.154203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.154471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.154609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.154786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.157740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.158025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.158152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.158282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.158366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.158557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.158697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.158838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.158914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.159092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.171975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.172246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.172372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.172524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.172607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.172780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.172921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.173026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.173166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.173332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.173460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.173611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.173696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.173933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.192682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.193100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.195959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.196394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.199215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.200012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.201144 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.201621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.202923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.203195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.203408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.203607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.203694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.203985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.210722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.211196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.212349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.212906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.215558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.215853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.216120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.216247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.216384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.216483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.216652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.216960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.225547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.225830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.226098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.226242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.226465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.226730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.233911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.234185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.234455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.234580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:27:14.234721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.234802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.234964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.235090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.235263 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.235390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.235538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.235618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.235726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.235833 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.236107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.236235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.236386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.236518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.236592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.236790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.236916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.237000 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.237089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.237286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.237413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.237549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.237798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.238068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:27:14.274710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f17fb5d9c0 pid: 80416 (default app) Jul 13 20:27:14.275448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f17fb5d9c0 pid: 80415 (default app) Jul 13 20:27:14.278102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa38571-4957-4f3a-83ff-107d23abfc76 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:14.278390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa38571-4957-4f3a-83ff-107d23abfc76 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:14.278668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa38571-4957-4f3a-83ff-107d23abfc76 None None] PATH_INFO: `/v3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:14.282997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Wed Jul 13 20:27:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:21.991078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506280a-79fd-4688-a876-77304f1868a8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:21.992055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506280a-79fd-4688-a876-77304f1868a8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:21.992055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506280a-79fd-4688-a876-77304f1868a8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:21.999209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:21 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:22.005393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e294890-fd85-4d71-885f-146e060784ba None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:22.005826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e294890-fd85-4d71-885f-146e060784ba None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:22.005972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e294890-fd85-4d71-885f-146e060784ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:22.083359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: INFO dbcounter [None req-9e294890-fd85-4d71-885f-146e060784ba None None] Registered counter for database keystone Jul 13 20:27:22.108141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG oslo_db.sqlalchemy.engines [None req-9e294890-fd85-4d71-885f-146e060784ba None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80416) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 20:27:22.119815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writer thread running {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 20:27:22.299143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG passlib.handlers.bcrypt [None req-9e294890-fd85-4d71-885f-146e060784ba None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80416) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 20:27:22.299500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG passlib.handlers.bcrypt [None req-9e294890-fd85-4d71-885f-146e060784ba None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80416) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 20:27:22.354724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9e294890-fd85-4d71-885f-146e060784ba None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:22.356382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9e294890-fd85-4d71-885f-146e060784ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:22.482477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 13 20:27:22.709156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506280a-79fd-4688-a876-77304f1868a8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:22.782586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: INFO dbcounter [None req-8506280a-79fd-4688-a876-77304f1868a8 None None] Registered counter for database keystone Jul 13 20:27:22.804578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG oslo_db.sqlalchemy.engines [None req-8506280a-79fd-4688-a876-77304f1868a8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80415) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 20:27:22.816630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writer thread running {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 20:27:22.939578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506280a-79fd-4688-a876-77304f1868a8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:22.945066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a64ce-885a-46b2-ae6b-1bfe3c8b973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:22.945732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a64ce-885a-46b2-ae6b-1bfe3c8b973e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:22.946014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a64ce-885a-46b2-ae6b-1bfe3c8b973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:22.946295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a64ce-885a-46b2-ae6b-1bfe3c8b973e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:22.955349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a64ce-885a-46b2-ae6b-1bfe3c8b973e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:23.007063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d86a64ce-885a-46b2-ae6b-1bfe3c8b973e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 20:27:23.007689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a64ce-885a-46b2-ae6b-1bfe3c8b973e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:23.017603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Wed Jul 13 20:27:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 20:27:23.024750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e294890-fd85-4d71-885f-146e060784ba None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:23.047289 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e294890-fd85-4d71-885f-146e060784ba None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:23.053093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb23cbf-28b2-49a8-9435-e306e221cd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:23.054236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb23cbf-28b2-49a8-9435-e306e221cd2c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:23.054523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb23cbf-28b2-49a8-9435-e306e221cd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:23.054795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb23cbf-28b2-49a8-9435-e306e221cd2c admin admin] PATH_INFO: `/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:23.056532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb23cbf-28b2-49a8-9435-e306e221cd2c admin admin] RBAC: Authorizing `identity:get_project(project_id=81ec70650e114b9b8d48f50c1f3f4abe)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:23.109952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ceb23cbf-28b2-49a8-9435-e306e221cd2c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 20:27:23.110285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb23cbf-28b2-49a8-9435-e306e221cd2c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:23.114982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Wed Jul 13 20:27:23 2022] GET /identity/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe => generated 344 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:25.114033 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a55de-5d92-4d8b-9596-63471c96ad84 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:25.114848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a55de-5d92-4d8b-9596-63471c96ad84 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:25.114988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a55de-5d92-4d8b-9596-63471c96ad84 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:25.121703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:25.127373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a742b-3a1e-43dc-8e33-534c37c431d0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:25.127816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a742b-3a1e-43dc-8e33-534c37c431d0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:25.127954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a742b-3a1e-43dc-8e33-534c37c431d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:25.148718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9b5a742b-3a1e-43dc-8e33-534c37c431d0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:25.150241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9b5a742b-3a1e-43dc-8e33-534c37c431d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:25.158719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 13 20:27:25.181225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a55de-5d92-4d8b-9596-63471c96ad84 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:25.203275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a55de-5d92-4d8b-9596-63471c96ad84 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:25.208554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695e103-0419-485a-b6b7-fdc46bff0984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:25.209193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695e103-0419-485a-b6b7-fdc46bff0984 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:25.209477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695e103-0419-485a-b6b7-fdc46bff0984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:25.209753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695e103-0419-485a-b6b7-fdc46bff0984 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:25.217756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695e103-0419-485a-b6b7-fdc46bff0984 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:25.219585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695e103-0419-485a-b6b7-fdc46bff0984 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:25.228410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Wed Jul 13 20:27:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:27:25.233827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a742b-3a1e-43dc-8e33-534c37c431d0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:25.243985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a742b-3a1e-43dc-8e33-534c37c431d0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:25.249425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa502a-673b-412a-b5dc-f4c4ab801522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:25.250284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa502a-673b-412a-b5dc-f4c4ab801522 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:25.250567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa502a-673b-412a-b5dc-f4c4ab801522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:25.250843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa502a-673b-412a-b5dc-f4c4ab801522 admin admin] PATH_INFO: `/v3/users/645d7df5139643969242f8464ac102fe` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:25.253029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa502a-673b-412a-b5dc-f4c4ab801522 admin admin] RBAC: Authorizing `identity:get_user(user_id=645d7df5139643969242f8464ac102fe)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:25.256025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa502a-673b-412a-b5dc-f4c4ab801522 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:25.258523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Wed Jul 13 20:27:25 2022] GET /identity/v3/users/645d7df5139643969242f8464ac102fe => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:27.326532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e81f-1d18-426f-a3d9-a0d5d2b0b18d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:27.327097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e81f-1d18-426f-a3d9-a0d5d2b0b18d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:27.327097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e81f-1d18-426f-a3d9-a0d5d2b0b18d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:27.328441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:27.333855 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3abb9-f9eb-4b33-9728-de8c2a0ae365 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:27.334281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3abb9-f9eb-4b33-9728-de8c2a0ae365 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:27.334458 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3abb9-f9eb-4b33-9728-de8c2a0ae365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:27.335532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ef8e0-0ade-489f-b5ad-470af557ab39 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:27.335963 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ef8e0-0ade-489f-b5ad-470af557ab39 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:27.336402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ef8e0-0ade-489f-b5ad-470af557ab39 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:27.338347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:27.343496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386930a-211c-401f-8302-080260ec1137 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:27.343900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386930a-211c-401f-8302-080260ec1137 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:27.344321 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386930a-211c-401f-8302-080260ec1137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:27.358479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4cd3abb9-f9eb-4b33-9728-de8c2a0ae365 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:27.359707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4cd3abb9-f9eb-4b33-9728-de8c2a0ae365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:27.389918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG passlib.handlers.bcrypt [None req-1386930a-211c-401f-8302-080260ec1137 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80415) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 20:27:27.390342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG passlib.handlers.bcrypt [None req-1386930a-211c-401f-8302-080260ec1137 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80415) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 20:27:27.435271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1386930a-211c-401f-8302-080260ec1137 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:27.436975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1386930a-211c-401f-8302-080260ec1137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:27.446404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:27.469783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3abb9-f9eb-4b33-9728-de8c2a0ae365 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:27.493876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:27.538523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:27.560360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386930a-211c-401f-8302-080260ec1137 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:27.572828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3abb9-f9eb-4b33-9728-de8c2a0ae365 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:27.607328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db039078-6e3f-4d67-8be5-a8ff6820c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:27.609329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039078-6e3f-4d67-8be5-a8ff6820c993 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:27.609329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039078-6e3f-4d67-8be5-a8ff6820c993 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:27.609329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039078-6e3f-4d67-8be5-a8ff6820c993 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:27.621959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db039078-6e3f-4d67-8be5-a8ff6820c993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-db039078-6e3f-4d67-8be5-a8ff6820c993 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:27.633316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:27.634198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:27.634994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:27:27.640519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db039078-6e3f-4d67-8be5-a8ff6820c993 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:27.661257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386930a-211c-401f-8302-080260ec1137 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:27.663653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:27.700690 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f51a72-30df-426a-8b07-2dc66549ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:27.701540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f51a72-30df-426a-8b07-2dc66549ac7e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:27.701853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f51a72-30df-426a-8b07-2dc66549ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:27.702136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f51a72-30df-426a-8b07-2dc66549ac7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:27.720452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:27.737913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db039078-6e3f-4d67-8be5-a8ff6820c993 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:27.772273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f51a72-30df-426a-8b07-2dc66549ac7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:27.773295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329e120-6fae-4eea-98c6-eef32672975f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:27.773835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f51a72-30df-426a-8b07-2dc66549ac7e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:27.773982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329e120-6fae-4eea-98c6-eef32672975f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:27.774270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329e120-6fae-4eea-98c6-eef32672975f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:27.774551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329e120-6fae-4eea-98c6-eef32672975f None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:27.775311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329e120-6fae-4eea-98c6-eef32672975f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:27.775656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329e120-6fae-4eea-98c6-eef32672975f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:27.777065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329e120-6fae-4eea-98c6-eef32672975f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:27.789033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:27.794606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329e120-6fae-4eea-98c6-eef32672975f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:27.873100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 20:27:27.878940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f51a72-30df-426a-8b07-2dc66549ac7e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:27.881798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:27.895876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329e120-6fae-4eea-98c6-eef32672975f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:27.928318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953dbe8-a0d4-4ec6-af42-cb9041e99688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:27.929197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953dbe8-a0d4-4ec6-af42-cb9041e99688 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:27.929477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953dbe8-a0d4-4ec6-af42-cb9041e99688 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:27.929750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953dbe8-a0d4-4ec6-af42-cb9041e99688 None admin] PATH_INFO: `/v3/users/645d7df5139643969242f8464ac102fe` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:27.930928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953dbe8-a0d4-4ec6-af42-cb9041e99688 None admin] RBAC: Authorizing `identity:get_user(user_id=645d7df5139643969242f8464ac102fe)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:27.932455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953dbe8-a0d4-4ec6-af42-cb9041e99688 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:27.934273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:27 2022] GET /identity/v3/users/645d7df5139643969242f8464ac102fe => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:27.939610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953dbe8-a0d4-4ec6-af42-cb9041e99688 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:27.954569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:27.981070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f51a72-30df-426a-8b07-2dc66549ac7e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:28.017369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80536e7-135c-4c9d-a6a4-5276485d9161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:28.018174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80536e7-135c-4c9d-a6a4-5276485d9161 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:28.018465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80536e7-135c-4c9d-a6a4-5276485d9161 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:28.018747 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80536e7-135c-4c9d-a6a4-5276485d9161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:28.036926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80536e7-135c-4c9d-a6a4-5276485d9161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:28.038394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80536e7-135c-4c9d-a6a4-5276485d9161 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:28.040671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953dbe8-a0d4-4ec6-af42-cb9041e99688 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f80536e7-135c-4c9d-a6a4-5276485d9161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:27:28.059082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:28.060081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:28.060861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:27:28.060861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:27:28.060861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:27:28.060861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:28.060861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 9/18] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:27:28.066543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80536e7-135c-4c9d-a6a4-5276485d9161 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:28.081372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46168230-d6c2-49f5-af86-ca73053ccd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:28.082188 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46168230-d6c2-49f5-af86-ca73053ccd9e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:28.082473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46168230-d6c2-49f5-af86-ca73053ccd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:28.083025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46168230-d6c2-49f5-af86-ca73053ccd9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:28.083998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46168230-d6c2-49f5-af86-ca73053ccd9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:28.085488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46168230-d6c2-49f5-af86-ca73053ccd9e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:28.087097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 10/19] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:28.093210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46168230-d6c2-49f5-af86-ca73053ccd9e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:28.124280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:28.167871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80536e7-135c-4c9d-a6a4-5276485d9161 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:28.172627 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:28.191851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46168230-d6c2-49f5-af86-ca73053ccd9e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:28.202209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10652c-3d20-40b4-acbd-e7355229a569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:28.202883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10652c-3d20-40b4-acbd-e7355229a569 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:28.203172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10652c-3d20-40b4-acbd-e7355229a569 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:28.203464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10652c-3d20-40b4-acbd-e7355229a569 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:28.204218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10652c-3d20-40b4-acbd-e7355229a569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:28.204536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10652c-3d20-40b4-acbd-e7355229a569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:28.205951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10652c-3d20-40b4-acbd-e7355229a569 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:28.223177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 10/20] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:28.224916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8252f3f-78bb-43a9-be70-77c0a7971ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:28.225571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8252f3f-78bb-43a9-be70-77c0a7971ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:28.225858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8252f3f-78bb-43a9-be70-77c0a7971ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:28.226135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8252f3f-78bb-43a9-be70-77c0a7971ab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:28.226976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8252f3f-78bb-43a9-be70-77c0a7971ab9 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.domain.id=default, user.id=645d7df5139643969242f8464ac102fe) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:28.227290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8252f3f-78bb-43a9-be70-77c0a7971ab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:28.228804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8252f3f-78bb-43a9-be70-77c0a7971ab9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:28.236878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 11/21] 10.222.0.47 () {56 vars in 1322 bytes} [Wed Jul 13 20:27:28 2022] GET /identity/v3/role_assignments?user.id=645d7df5139643969242f8464ac102fe&scope.domain.id=default&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:30.503200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540288ba-d2a9-4a50-a52c-3c444a760b04 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:30.503750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540288ba-d2a9-4a50-a52c-3c444a760b04 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:30.503823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540288ba-d2a9-4a50-a52c-3c444a760b04 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:30.505168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 11/22] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:30.510426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b6f1-c162-4239-961c-b37034116c65 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:30.510832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b6f1-c162-4239-961c-b37034116c65 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:30.510978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b6f1-c162-4239-961c-b37034116c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:30.536447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5793b6f1-c162-4239-961c-b37034116c65 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:30.537895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5793b6f1-c162-4239-961c-b37034116c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:30.614103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 12/23] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:30.644861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540288ba-d2a9-4a50-a52c-3c444a760b04 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:30.768312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a237bb1-8ca4-413a-b77e-8afd91e0046f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:30.768804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a237bb1-8ca4-413a-b77e-8afd91e0046f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:30.769248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a237bb1-8ca4-413a-b77e-8afd91e0046f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:30.771093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 13/24] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:30.776485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3f23d-ccb3-4ae2-b92b-37c7402d0173 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:30.776944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3f23d-ccb3-4ae2-b92b-37c7402d0173 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:30.777357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3f23d-ccb3-4ae2-b92b-37c7402d0173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:30.804231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540288ba-d2a9-4a50-a52c-3c444a760b04 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:30.808089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2db3f23d-ccb3-4ae2-b92b-37c7402d0173 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:30.810220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2db3f23d-ccb3-4ae2-b92b-37c7402d0173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:30.850333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:30.896328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5319f4-47c7-4a8c-be64-93abb9ebdcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:30.897153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5319f4-47c7-4a8c-be64-93abb9ebdcce None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:30.897437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5319f4-47c7-4a8c-be64-93abb9ebdcce None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:30.897716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5319f4-47c7-4a8c-be64-93abb9ebdcce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:30.921867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5319f4-47c7-4a8c-be64-93abb9ebdcce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:30.923368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5319f4-47c7-4a8c-be64-93abb9ebdcce None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3b5319f4-47c7-4a8c-be64-93abb9ebdcce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:27:30.964286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:30.965136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:30.965921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:27:30.965921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:27:30.965921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:27:30.965921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:30.966115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 12/25] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:30.967217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a339dc-a1b6-49ce-bb20-f56e553194fb None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:30.967622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a339dc-a1b6-49ce-bb20-f56e553194fb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:30.967904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a339dc-a1b6-49ce-bb20-f56e553194fb None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:30.969160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 13/26] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:30.969426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 14/27] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:27:30.974602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff5883-a997-4151-9e15-a9d81c830669 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:30.974890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff5883-a997-4151-9e15-a9d81c830669 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:30.975166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff5883-a997-4151-9e15-a9d81c830669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:30.977166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3f23d-ccb3-4ae2-b92b-37c7402d0173 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:30.982551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:31.011509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4eff5883-a997-4151-9e15-a9d81c830669 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:31.012720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4eff5883-a997-4151-9e15-a9d81c830669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:31.114388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 14/28] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:27:31.115352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff5883-a997-4151-9e15-a9d81c830669 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:31.162278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3f23d-ccb3-4ae2-b92b-37c7402d0173 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:31.204052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbd262-e499-46c7-9040-f540617a6d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:31.205011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbd262-e499-46c7-9040-f540617a6d0e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.206029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbd262-e499-46c7-9040-f540617a6d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.206450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbd262-e499-46c7-9040-f540617a6d0e None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.207322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbd262-e499-46c7-9040-f540617a6d0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:31.208214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbd262-e499-46c7-9040-f540617a6d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:31.210465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbd262-e499-46c7-9040-f540617a6d0e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:31.228363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:31.233318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff5883-a997-4151-9e15-a9d81c830669 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:31.235663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 15/29] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:31.237108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413de7b-5656-4c90-8c95-77ec532891b6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.237533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413de7b-5656-4c90-8c95-77ec532891b6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.237968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413de7b-5656-4c90-8c95-77ec532891b6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.239336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 16/30] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:31.240200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413de7b-5656-4c90-8c95-77ec532891b6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:31.276720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c0c8d-468e-419c-b92a-839dfe567bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:31.277727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c0c8d-468e-419c-b92a-839dfe567bce None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.278165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c0c8d-468e-419c-b92a-839dfe567bce None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.278671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c0c8d-468e-419c-b92a-839dfe567bce None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.279689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c0c8d-468e-419c-b92a-839dfe567bce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:31.281252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c0c8d-468e-419c-b92a-839dfe567bce None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:31.314058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:31.354000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413de7b-5656-4c90-8c95-77ec532891b6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:31.388426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8eeee4-96d5-4082-a072-499fe8ba3e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:31.389637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8eeee4-96d5-4082-a072-499fe8ba3e05 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.390092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8eeee4-96d5-4082-a072-499fe8ba3e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.390529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8eeee4-96d5-4082-a072-499fe8ba3e05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.401043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8eeee4-96d5-4082-a072-499fe8ba3e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-de8eeee4-96d5-4082-a072-499fe8ba3e05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:31.408937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:31.409821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:31.410535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:31.413006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 17/31] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:27:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:31.413996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8eeee4-96d5-4082-a072-499fe8ba3e05 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:31.422517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 15/32] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 13 20:27:31 2022] POST /identity/v3/roles => generated 219 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:27:31.424330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486056d-96a8-429a-b17e-0693390a38f5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.424630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486056d-96a8-429a-b17e-0693390a38f5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.425084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486056d-96a8-429a-b17e-0693390a38f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.445453 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3486056d-96a8-429a-b17e-0693390a38f5 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:31.447237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3486056d-96a8-429a-b17e-0693390a38f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:31.527115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 16/33] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:31.527756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486056d-96a8-429a-b17e-0693390a38f5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:31.529484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8eeee4-96d5-4082-a072-499fe8ba3e05 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:31.558713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:31.578612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933d54b-adb8-43b1-9a5d-cddd95cbaa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:31.579766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933d54b-adb8-43b1-9a5d-cddd95cbaa1c None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.580250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933d54b-adb8-43b1-9a5d-cddd95cbaa1c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.580789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933d54b-adb8-43b1-9a5d-cddd95cbaa1c None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.581782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933d54b-adb8-43b1-9a5d-cddd95cbaa1c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:31.583750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933d54b-adb8-43b1-9a5d-cddd95cbaa1c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:31.637637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:31.661626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486056d-96a8-429a-b17e-0693390a38f5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:31.700082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041fadf-76f8-4259-913c-9a53d7440aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:31.700828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041fadf-76f8-4259-913c-9a53d7440aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.701126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041fadf-76f8-4259-913c-9a53d7440aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.701408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041fadf-76f8-4259-913c-9a53d7440aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.702226 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041fadf-76f8-4259-913c-9a53d7440aa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:31.702539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041fadf-76f8-4259-913c-9a53d7440aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:31.704186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041fadf-76f8-4259-913c-9a53d7440aa7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:31.718068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 17/34] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:31.718678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041fadf-76f8-4259-913c-9a53d7440aa7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:31.754734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 18/35] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:27:31 2022] POST /identity/v3/projects => generated 301 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:27:31.756008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933d54b-adb8-43b1-9a5d-cddd95cbaa1c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:31.873489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041fadf-76f8-4259-913c-9a53d7440aa7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:31.917953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9250e-3aac-4404-a354-9ec0ea5ef35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:31.918578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9250e-3aac-4404-a354-9ec0ea5ef35f None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.918873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9250e-3aac-4404-a354-9ec0ea5ef35f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.919158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9250e-3aac-4404-a354-9ec0ea5ef35f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.920021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9250e-3aac-4404-a354-9ec0ea5ef35f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:31.921555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9250e-3aac-4404-a354-9ec0ea5ef35f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:31.942515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 18/36] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 13 20:27:31 2022] POST /identity/v3/roles => generated 225 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:27:31.944774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00312d9-da8b-4679-bdf4-9638dcdbe9f1 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.945074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00312d9-da8b-4679-bdf4-9638dcdbe9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.945536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00312d9-da8b-4679-bdf4-9638dcdbe9f1 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.947837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 19/37] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:31.954437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86537d3f-fada-4de2-abbb-a2bd8a4264ee None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:31.954742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86537d3f-fada-4de2-abbb-a2bd8a4264ee None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:31.955026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86537d3f-fada-4de2-abbb-a2bd8a4264ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:31.973710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-86537d3f-fada-4de2-abbb-a2bd8a4264ee None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:31.975015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-86537d3f-fada-4de2-abbb-a2bd8a4264ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:31.980163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:31.998216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933d54b-adb8-43b1-9a5d-cddd95cbaa1c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:32.040417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed2a2-ffc2-4030-9d25-fd536c6eccc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:32.041783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed2a2-ffc2-4030-9d25-fd536c6eccc3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:32.042078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed2a2-ffc2-4030-9d25-fd536c6eccc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:32.042358 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed2a2-ffc2-4030-9d25-fd536c6eccc3 None admin] PATH_INFO: `/v3/users/645d7df5139643969242f8464ac102fe` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:32.043903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ed2a2-ffc2-4030-9d25-fd536c6eccc3 None admin] RBAC: Authorizing `identity:get_user(user_id=645d7df5139643969242f8464ac102fe)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:32.046329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ed2a2-ffc2-4030-9d25-fd536c6eccc3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:32.050426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 19/38] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:31 2022] GET /identity/v3/users/645d7df5139643969242f8464ac102fe => generated 247 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:32.055410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed2a2-ffc2-4030-9d25-fd536c6eccc3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:32.075607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 20/39] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:27:32.095299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86537d3f-fada-4de2-abbb-a2bd8a4264ee None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:32.117512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:32.191925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed2a2-ffc2-4030-9d25-fd536c6eccc3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:32.230848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86537d3f-fada-4de2-abbb-a2bd8a4264ee None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:32.238017 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c829818-e4e5-4dd0-99c0-d4d284bc46e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:32.238837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c829818-e4e5-4dd0-99c0-d4d284bc46e5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:32.239553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c829818-e4e5-4dd0-99c0-d4d284bc46e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:32.239553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c829818-e4e5-4dd0-99c0-d4d284bc46e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:32.240890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c829818-e4e5-4dd0-99c0-d4d284bc46e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:32.242499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c829818-e4e5-4dd0-99c0-d4d284bc46e5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:32.244405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 20/40] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:32.249991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c829818-e4e5-4dd0-99c0-d4d284bc46e5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:32.274811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506757b-3ecc-4b36-bbb1-fe4340049926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:32.275517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506757b-3ecc-4b36-bbb1-fe4340049926 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:32.275809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506757b-3ecc-4b36-bbb1-fe4340049926 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:32.276135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506757b-3ecc-4b36-bbb1-fe4340049926 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:32.276894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506757b-3ecc-4b36-bbb1-fe4340049926 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:32.278304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506757b-3ecc-4b36-bbb1-fe4340049926 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:32.288031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:32.299585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 21/41] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 13 20:27:32 2022] POST /identity/v3/roles => generated 223 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:27:32.366672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c829818-e4e5-4dd0-99c0-d4d284bc46e5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:32.381787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:32.392596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caff19a-1d58-4888-87e9-620bf906f6c2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:32.392920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caff19a-1d58-4888-87e9-620bf906f6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:32.393167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caff19a-1d58-4888-87e9-620bf906f6c2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:32.394572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 22/42] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:27:32.400080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623612-842d-43fe-ad97-6a723042370c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:32.400377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623612-842d-43fe-ad97-6a723042370c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:32.400675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623612-842d-43fe-ad97-6a723042370c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:32.406322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f772d-ae2f-4a44-b32d-ee18a44a1137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:32.407216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f772d-ae2f-4a44-b32d-ee18a44a1137 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:32.407557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f772d-ae2f-4a44-b32d-ee18a44a1137 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:32.407825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f772d-ae2f-4a44-b32d-ee18a44a1137 None admin] PATH_INFO: `/v3/domains/default/users/645d7df5139643969242f8464ac102fe/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:32.409998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f772d-ae2f-4a44-b32d-ee18a44a1137 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=645d7df5139643969242f8464ac102fe, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:32.411533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f772d-ae2f-4a44-b32d-ee18a44a1137 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:32.419276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d1623612-842d-43fe-ad97-6a723042370c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:32.420849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d1623612-842d-43fe-ad97-6a723042370c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:32.427501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 21/43] 10.222.0.47 () {58 vars in 1361 bytes} [Wed Jul 13 20:27:32 2022] PUT /identity/v3/domains/default/users/645d7df5139643969242f8464ac102fe/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:27:32.507746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 23/44] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:32.532075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f772d-ae2f-4a44-b32d-ee18a44a1137 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:32.640601 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f772d-ae2f-4a44-b32d-ee18a44a1137 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:32.674935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad199be9-d1d1-4711-acaf-15f2636e2c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:32.675789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad199be9-d1d1-4711-acaf-15f2636e2c13 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:32.676083 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad199be9-d1d1-4711-acaf-15f2636e2c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:32.676365 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad199be9-d1d1-4711-acaf-15f2636e2c13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:32.677552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad199be9-d1d1-4711-acaf-15f2636e2c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:32.679048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad199be9-d1d1-4711-acaf-15f2636e2c13 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:32.680725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 22/45] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:32.686715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1623612-842d-43fe-ad97-6a723042370c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:32.790122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1623612-842d-43fe-ad97-6a723042370c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:32.819359 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:32.825934 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97c2bd-c197-4f95-bc39-25692b6d8e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:32.826660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97c2bd-c197-4f95-bc39-25692b6d8e2c None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:32.826987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97c2bd-c197-4f95-bc39-25692b6d8e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:32.827269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97c2bd-c197-4f95-bc39-25692b6d8e2c None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:32.828300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97c2bd-c197-4f95-bc39-25692b6d8e2c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:32.829780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97c2bd-c197-4f95-bc39-25692b6d8e2c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:32.857153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 24/46] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:27:32 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:27:34.458619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac0b9f-c5aa-4eb8-9288-0f9d7df88aab None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:34.459130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac0b9f-c5aa-4eb8-9288-0f9d7df88aab None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:34.459130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac0b9f-c5aa-4eb8-9288-0f9d7df88aab None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:34.460558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 23/47] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:34.466034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea940f5-5e65-41e2-9714-a558050e51d1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:34.466438 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea940f5-5e65-41e2-9714-a558050e51d1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:34.466608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea940f5-5e65-41e2-9714-a558050e51d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:34.485172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dea940f5-5e65-41e2-9714-a558050e51d1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:34.486388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dea940f5-5e65-41e2-9714-a558050e51d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:34.662375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 25/48] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:34.685828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac0b9f-c5aa-4eb8-9288-0f9d7df88aab None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:34.732425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:34.792167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac0b9f-c5aa-4eb8-9288-0f9d7df88aab None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:34.827600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f240b2-344e-4529-a680-6ae7920d3e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:34.828496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f240b2-344e-4529-a680-6ae7920d3e92 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:34.828787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f240b2-344e-4529-a680-6ae7920d3e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:34.829066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f240b2-344e-4529-a680-6ae7920d3e92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:34.839216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f240b2-344e-4529-a680-6ae7920d3e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-18f240b2-344e-4529-a680-6ae7920d3e92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:34.846830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:34.847991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:34.849008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 24/49] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:27:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:34.856173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea940f5-5e65-41e2-9714-a558050e51d1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:34.874408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1cd23-ab5c-4016-9a11-962f34d99e13 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:34.874668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1cd23-ab5c-4016-9a11-962f34d99e13 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:34.874974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1cd23-ab5c-4016-9a11-962f34d99e13 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:34.876353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 25/50] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:27:34.881502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281bd88-774c-44e4-8789-ad26a56ee235 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:34.881801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281bd88-774c-44e4-8789-ad26a56ee235 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:34.882090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281bd88-774c-44e4-8789-ad26a56ee235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:34.899774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2281bd88-774c-44e4-8789-ad26a56ee235 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:34.901291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2281bd88-774c-44e4-8789-ad26a56ee235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:34.959974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea940f5-5e65-41e2-9714-a558050e51d1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:34.974607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:34.976402 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 26/51] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:35.000330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee925d61-6c57-4de1-9d36-4ed0f1af2f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:35.000933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925d61-6c57-4de1-9d36-4ed0f1af2f86 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:35.001220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925d61-6c57-4de1-9d36-4ed0f1af2f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:35.001503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925d61-6c57-4de1-9d36-4ed0f1af2f86 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:35.002268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee925d61-6c57-4de1-9d36-4ed0f1af2f86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:35.002579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee925d61-6c57-4de1-9d36-4ed0f1af2f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:35.004094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281bd88-774c-44e4-8789-ad26a56ee235 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:35.004246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee925d61-6c57-4de1-9d36-4ed0f1af2f86 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:35.013155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 26/52] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:35.020178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee925d61-6c57-4de1-9d36-4ed0f1af2f86 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:35.035348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:35.104566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281bd88-774c-44e4-8789-ad26a56ee235 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:35.126735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee925d61-6c57-4de1-9d36-4ed0f1af2f86 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:35.137814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd852d-0af8-4e5a-b2b1-7dc1f4b69e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:35.138625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd852d-0af8-4e5a-b2b1-7dc1f4b69e23 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:35.138914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd852d-0af8-4e5a-b2b1-7dc1f4b69e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:35.139327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd852d-0af8-4e5a-b2b1-7dc1f4b69e23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:35.140537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd852d-0af8-4e5a-b2b1-7dc1f4b69e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:35.142100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd852d-0af8-4e5a-b2b1-7dc1f4b69e23 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:35.143781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 27/53] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:35.150226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd852d-0af8-4e5a-b2b1-7dc1f4b69e23 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:35.159658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c03b9f-ef4d-46e4-b7bd-9400ed35b03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:35.160550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c03b9f-ef4d-46e4-b7bd-9400ed35b03e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:35.160834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c03b9f-ef4d-46e4-b7bd-9400ed35b03e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:35.161111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c03b9f-ef4d-46e4-b7bd-9400ed35b03e None admin] PATH_INFO: `/v3/users/645d7df5139643969242f8464ac102fe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:35.162538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c03b9f-ef4d-46e4-b7bd-9400ed35b03e None admin] RBAC: Authorizing `identity:get_user(user_id=645d7df5139643969242f8464ac102fe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:35.164023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c03b9f-ef4d-46e4-b7bd-9400ed35b03e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:35.165651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 27/54] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:35 2022] GET /identity/v3/users/645d7df5139643969242f8464ac102fe => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:35.170930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c03b9f-ef4d-46e4-b7bd-9400ed35b03e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:35.214364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:35.247318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:35.252043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd852d-0af8-4e5a-b2b1-7dc1f4b69e23 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:35.276530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c03b9f-ef4d-46e4-b7bd-9400ed35b03e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:35.289916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171cbea-80b9-4322-85fe-74b3471b6a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:35.290594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171cbea-80b9-4322-85fe-74b3471b6a20 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:35.290886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171cbea-80b9-4322-85fe-74b3471b6a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:35.291168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171cbea-80b9-4322-85fe-74b3471b6a20 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:35.291927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171cbea-80b9-4322-85fe-74b3471b6a20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:35.293420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171cbea-80b9-4322-85fe-74b3471b6a20 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:35.313189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f121e-7e78-4b72-82df-800fd904f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:35.314093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f121e-7e78-4b72-82df-800fd904f733 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:35.314384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f121e-7e78-4b72-82df-800fd904f733 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:35.314708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f121e-7e78-4b72-82df-800fd904f733 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:35.316089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f121e-7e78-4b72-82df-800fd904f733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:35.317572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f121e-7e78-4b72-82df-800fd904f733 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:35.319211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 28/55] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:35.321691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 28/56] 10.222.0.47 () {60 vars in 1140 bytes} [Wed Jul 13 20:27:35 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:27:35.325521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f121e-7e78-4b72-82df-800fd904f733 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:35.437809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f121e-7e78-4b72-82df-800fd904f733 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:35.449849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:35.476979 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515beec-fa78-4236-8dfa-b7d244d37895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:35.477652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515beec-fa78-4236-8dfa-b7d244d37895 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:35.477936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515beec-fa78-4236-8dfa-b7d244d37895 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:35.478215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515beec-fa78-4236-8dfa-b7d244d37895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:35.479058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515beec-fa78-4236-8dfa-b7d244d37895 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.domain.id=default, user.id=645d7df5139643969242f8464ac102fe) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:35.479372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515beec-fa78-4236-8dfa-b7d244d37895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:35.480841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515beec-fa78-4236-8dfa-b7d244d37895 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:35.489760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 29/57] 10.222.0.47 () {56 vars in 1322 bytes} [Wed Jul 13 20:27:35 2022] GET /identity/v3/role_assignments?user.id=645d7df5139643969242f8464ac102fe&scope.domain.id=default&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 530 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:37.308553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b274d5-6c3b-456f-afed-9d2f95aeceda None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:37.309085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b274d5-6c3b-456f-afed-9d2f95aeceda None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:37.309085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b274d5-6c3b-456f-afed-9d2f95aeceda None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:37.310512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 29/58] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:37.317089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45012577-de92-4285-9b18-1b8b15ad6c32 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:37.317744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45012577-de92-4285-9b18-1b8b15ad6c32 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:37.317874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45012577-de92-4285-9b18-1b8b15ad6c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:37.343496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-45012577-de92-4285-9b18-1b8b15ad6c32 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:37.344742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-45012577-de92-4285-9b18-1b8b15ad6c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:37.418089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 30/59] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:37.443100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b274d5-6c3b-456f-afed-9d2f95aeceda None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:37.556591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b274d5-6c3b-456f-afed-9d2f95aeceda None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:37.592779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd9ceb-8c75-47b7-8866-458951f5e640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:37.593513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd9ceb-8c75-47b7-8866-458951f5e640 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:37.593805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd9ceb-8c75-47b7-8866-458951f5e640 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:37.594086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd9ceb-8c75-47b7-8866-458951f5e640 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:37.595316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd9ceb-8c75-47b7-8866-458951f5e640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:37.596828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd9ceb-8c75-47b7-8866-458951f5e640 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:37.598434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 30/60] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:37.607054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45012577-de92-4285-9b18-1b8b15ad6c32 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:37.717426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45012577-de92-4285-9b18-1b8b15ad6c32 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:37.719979 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:37.752880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22c2bb-a7ef-4d33-a36b-9d2f030215b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:37.753525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22c2bb-a7ef-4d33-a36b-9d2f030215b8 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:37.753898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22c2bb-a7ef-4d33-a36b-9d2f030215b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:37.754189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22c2bb-a7ef-4d33-a36b-9d2f030215b8 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:37.754911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22c2bb-a7ef-4d33-a36b-9d2f030215b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:37.756505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22c2bb-a7ef-4d33-a36b-9d2f030215b8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:37.784719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 31/61] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:27:37 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:27:40.912482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056c5e-fa22-4d63-a214-dc4a65b1d39b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:40.913520 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056c5e-fa22-4d63-a214-dc4a65b1d39b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:40.914031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056c5e-fa22-4d63-a214-dc4a65b1d39b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:40.916143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 31/62] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:40.921192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5157a-1f77-49c1-8637-866bd241a3a8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:40.922035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5157a-1f77-49c1-8637-866bd241a3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:40.922690 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5157a-1f77-49c1-8637-866bd241a3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:40.943164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-86b5157a-1f77-49c1-8637-866bd241a3a8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:40.944716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-86b5157a-1f77-49c1-8637-866bd241a3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:41.029126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 32/63] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:41.065901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2056c5e-fa22-4d63-a214-dc4a65b1d39b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:41.094033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:41.114203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5745d-9b69-4dbf-a0c8-ce51e9b8eaa3 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.114703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5745d-9b69-4dbf-a0c8-ce51e9b8eaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.115140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5745d-9b69-4dbf-a0c8-ce51e9b8eaa3 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.116774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 33/64] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:41.121919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32a53a-8629-4813-a2c7-2dfcf98df66a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.122425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32a53a-8629-4813-a2c7-2dfcf98df66a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.122852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32a53a-8629-4813-a2c7-2dfcf98df66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.159143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3f32a53a-8629-4813-a2c7-2dfcf98df66a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:41.160521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3f32a53a-8629-4813-a2c7-2dfcf98df66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:41.229219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2056c5e-fa22-4d63-a214-dc4a65b1d39b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:41.259043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 34/65] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:27:41.260735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e79d5b-e2d6-4d4b-a19e-13a0a811c7e6 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.261231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e79d5b-e2d6-4d4b-a19e-13a0a811c7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.261706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e79d5b-e2d6-4d4b-a19e-13a0a811c7e6 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.267376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 35/66] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:41.268759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fb327-8375-4703-829a-ddc53fb7107b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.269291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fb327-8375-4703-829a-ddc53fb7107b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.269765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fb327-8375-4703-829a-ddc53fb7107b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.285113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f644-983d-44b2-b67e-7cf1790e364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:41.285930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f644-983d-44b2-b67e-7cf1790e364a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.286219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f644-983d-44b2-b67e-7cf1790e364a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.286512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f644-983d-44b2-b67e-7cf1790e364a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.289021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7b7fb327-8375-4703-829a-ddc53fb7107b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:41.290434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7b7fb327-8375-4703-829a-ddc53fb7107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:41.304211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766f644-983d-44b2-b67e-7cf1790e364a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:41.306471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a766f644-983d-44b2-b67e-7cf1790e364a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:41.312317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:41.313214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:41.314455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 32/67] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:41.315436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64553b72-c496-421a-854a-b590e3f14920 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.315553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64553b72-c496-421a-854a-b590e3f14920 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.315798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64553b72-c496-421a-854a-b590e3f14920 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.317049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 33/68] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:41.319818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64553b72-c496-421a-854a-b590e3f14920 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:41.367841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 36/69] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:41.368697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fb327-8375-4703-829a-ddc53fb7107b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:41.411970 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:41.436340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64553b72-c496-421a-854a-b590e3f14920 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:41.471276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fb327-8375-4703-829a-ddc53fb7107b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:41.472759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79626-7c41-45f3-97ff-f67fc0fb62cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:41.473581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79626-7c41-45f3-97ff-f67fc0fb62cf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.473870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79626-7c41-45f3-97ff-f67fc0fb62cf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.474158 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79626-7c41-45f3-97ff-f67fc0fb62cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.484764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd79626-7c41-45f3-97ff-f67fc0fb62cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5bd79626-7c41-45f3-97ff-f67fc0fb62cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:41.492532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:41.493349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:41.494360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 34/70] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:41.495033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb9dd6-1421-4bcd-84e1-15fdc70c6d7e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.495247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb9dd6-1421-4bcd-84e1-15fdc70c6d7e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.495536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb9dd6-1421-4bcd-84e1-15fdc70c6d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.510679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25667a25-92b2-4a6d-8653-46454f95bbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:41.511363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25667a25-92b2-4a6d-8653-46454f95bbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.511695 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25667a25-92b2-4a6d-8653-46454f95bbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.511984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25667a25-92b2-4a6d-8653-46454f95bbe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.512720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25667a25-92b2-4a6d-8653-46454f95bbe9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:41.513024 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25667a25-92b2-4a6d-8653-46454f95bbe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:41.513119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e7bb9dd6-1421-4bcd-84e1-15fdc70c6d7e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:41.514342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e7bb9dd6-1421-4bcd-84e1-15fdc70c6d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:41.514455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25667a25-92b2-4a6d-8653-46454f95bbe9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:41.523169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 37/71] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:41.523976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25667a25-92b2-4a6d-8653-46454f95bbe9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:41.538851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:41.581146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 35/72] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:41.582280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccf897-691f-4ee2-8a4d-1273ffd244ff None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.582581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccf897-691f-4ee2-8a4d-1273ffd244ff None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.582868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccf897-691f-4ee2-8a4d-1273ffd244ff None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.584203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 36/73] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:41.588620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccf897-691f-4ee2-8a4d-1273ffd244ff None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:41.628719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25667a25-92b2-4a6d-8653-46454f95bbe9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:41.653151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:41.662238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532cb72-deba-471c-b5c4-b51806606321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:41.663052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532cb72-deba-471c-b5c4-b51806606321 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.663341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532cb72-deba-471c-b5c4-b51806606321 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.663661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532cb72-deba-471c-b5c4-b51806606321 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.675427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532cb72-deba-471c-b5c4-b51806606321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5532cb72-deba-471c-b5c4-b51806606321 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:41.684595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:41.685446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:41.686330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 38/74] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:41.686937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccf897-691f-4ee2-8a4d-1273ffd244ff None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:41.687008 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532cb72-deba-471c-b5c4-b51806606321 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:41.725446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04dd29-4c82-4d43-934e-5a13c196f310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:41.726134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04dd29-4c82-4d43-934e-5a13c196f310 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.726430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04dd29-4c82-4d43-934e-5a13c196f310 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.726718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04dd29-4c82-4d43-934e-5a13c196f310 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.727507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04dd29-4c82-4d43-934e-5a13c196f310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:41.727826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04dd29-4c82-4d43-934e-5a13c196f310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:41.729255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04dd29-4c82-4d43-934e-5a13c196f310 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:41.737795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 37/75] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:41.739278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcd60f-89ec-430f-9892-fddf0c278b3d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.739607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcd60f-89ec-430f-9892-fddf0c278b3d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.739897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcd60f-89ec-430f-9892-fddf0c278b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.757208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4dfcd60f-89ec-430f-9892-fddf0c278b3d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:41.758408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4dfcd60f-89ec-430f-9892-fddf0c278b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:41.762651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:41.793473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532cb72-deba-471c-b5c4-b51806606321 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:41.823136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 38/76] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:41.823995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcd60f-89ec-430f-9892-fddf0c278b3d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:41.827992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb680589-085f-4348-a5bb-e7f79ab74235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:41.828886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb680589-085f-4348-a5bb-e7f79ab74235 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.829173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb680589-085f-4348-a5bb-e7f79ab74235 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.829455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb680589-085f-4348-a5bb-e7f79ab74235 None admin] PATH_INFO: `/v3/users/6d22fba8da874faab7222240ee9e9e8c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.844889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb680589-085f-4348-a5bb-e7f79ab74235 None admin] RBAC: Authorizing `identity:get_user(user_id=6d22fba8da874faab7222240ee9e9e8c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:41.846481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb680589-085f-4348-a5bb-e7f79ab74235 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:41.848356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 39/77] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/users/6d22fba8da874faab7222240ee9e9e8c => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:41.849032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb680589-085f-4348-a5bb-e7f79ab74235 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:41.898635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:41.939361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcd60f-89ec-430f-9892-fddf0c278b3d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:41.948580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb680589-085f-4348-a5bb-e7f79ab74235 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:41.973363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1f3af-5d49-40c1-9226-aa9522694de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:41.974182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1f3af-5d49-40c1-9226-aa9522694de7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.974472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1f3af-5d49-40c1-9226-aa9522694de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.974748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1f3af-5d49-40c1-9226-aa9522694de7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.975950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:41.982523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9f3c7-b73b-4a68-9c71-48a3a78faf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:41.983209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9f3c7-b73b-4a68-9c71-48a3a78faf09 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:41.983542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9f3c7-b73b-4a68-9c71-48a3a78faf09 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:41.983831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9f3c7-b73b-4a68-9c71-48a3a78faf09 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:41.984594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9f3c7-b73b-4a68-9c71-48a3a78faf09 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:41.984910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9f3c7-b73b-4a68-9c71-48a3a78faf09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:41.985232 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1f3af-5d49-40c1-9226-aa9522694de7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:41.986323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9f3c7-b73b-4a68-9c71-48a3a78faf09 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2ff1f3af-5d49-40c1-9226-aa9522694de7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:41.993352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:41.994157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:27:41.994974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:41.996098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 40/78] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:41.996098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9f3c7-b73b-4a68-9c71-48a3a78faf09 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:41.999089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 39/79] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:41.999755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1f3af-5d49-40c1-9226-aa9522694de7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:42.097252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9f3c7-b73b-4a68-9c71-48a3a78faf09 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:42.100921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1f3af-5d49-40c1-9226-aa9522694de7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:42.120064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:42.130625 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0f5c1-d138-4e00-be08-76f513728814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:42.131566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0f5c1-d138-4e00-be08-76f513728814 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:42.131858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0f5c1-d138-4e00-be08-76f513728814 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:42.132139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0f5c1-d138-4e00-be08-76f513728814 None admin] PATH_INFO: `/v3/users/645d7df5139643969242f8464ac102fe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:42.133354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0f5c1-d138-4e00-be08-76f513728814 None admin] RBAC: Authorizing `identity:get_user(user_id=645d7df5139643969242f8464ac102fe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:42.134813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0f5c1-d138-4e00-be08-76f513728814 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:42.135167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336c039-8935-4217-8e8e-48c94047ad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:42.136035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336c039-8935-4217-8e8e-48c94047ad8b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:42.136332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336c039-8935-4217-8e8e-48c94047ad8b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:42.136495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 41/80] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/users/645d7df5139643969242f8464ac102fe => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:42.136938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336c039-8935-4217-8e8e-48c94047ad8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:42.137268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0f5c1-d138-4e00-be08-76f513728814 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:42.138159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336c039-8935-4217-8e8e-48c94047ad8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:42.139704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336c039-8935-4217-8e8e-48c94047ad8b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:42.141335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 40/81] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:42.142098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336c039-8935-4217-8e8e-48c94047ad8b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:42.198668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:42.241086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336c039-8935-4217-8e8e-48c94047ad8b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:42.241285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0f5c1-d138-4e00-be08-76f513728814 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:42.274361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f6731-1a78-48b7-bca5-880123566e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:42.274507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8005c-8f56-4dfd-a146-a4c84beb3725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:42.274956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8005c-8f56-4dfd-a146-a4c84beb3725 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:42.275171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f6731-1a78-48b7-bca5-880123566e62 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:42.275261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8005c-8f56-4dfd-a146-a4c84beb3725 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:42.275483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f6731-1a78-48b7-bca5-880123566e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:42.275573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8005c-8f56-4dfd-a146-a4c84beb3725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:42.275762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f6731-1a78-48b7-bca5-880123566e62 None admin] PATH_INFO: `/v3/users/6d22fba8da874faab7222240ee9e9e8c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:42.276945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f6731-1a78-48b7-bca5-880123566e62 None admin] RBAC: Authorizing `identity:get_user(user_id=6d22fba8da874faab7222240ee9e9e8c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:42.278402 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f6731-1a78-48b7-bca5-880123566e62 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:42.280556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 41/82] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:42 2022] GET /identity/v3/users/6d22fba8da874faab7222240ee9e9e8c => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:42.281496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f6731-1a78-48b7-bca5-880123566e62 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:42.339316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:42.341738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8005c-8f56-4dfd-a146-a4c84beb3725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:42.343318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8005c-8f56-4dfd-a146-a4c84beb3725 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:42.382357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f6731-1a78-48b7-bca5-880123566e62 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:42.399130 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:42.415311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9173de0-31bc-49e0-9239-6685460e4ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:42.416026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9173de0-31bc-49e0-9239-6685460e4ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:42.416317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9173de0-31bc-49e0-9239-6685460e4ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:42.416621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9173de0-31bc-49e0-9239-6685460e4ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:42.417361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9173de0-31bc-49e0-9239-6685460e4ee8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:42.417676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9173de0-31bc-49e0-9239-6685460e4ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:42.419077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9173de0-31bc-49e0-9239-6685460e4ee8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:42.428092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 42/83] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:27:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:42.428757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9173de0-31bc-49e0-9239-6685460e4ee8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:42.443300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 42/84] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:42.444403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8005c-8f56-4dfd-a146-a4c84beb3725 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:42.535338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9173de0-31bc-49e0-9239-6685460e4ee8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:42.549887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8005c-8f56-4dfd-a146-a4c84beb3725 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:42.552390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:42.571761 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914d2e6-d5af-489e-b235-796cb3a5aeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:42.572416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914d2e6-d5af-489e-b235-796cb3a5aeac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:42.572761 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914d2e6-d5af-489e-b235-796cb3a5aeac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:42.572968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914d2e6-d5af-489e-b235-796cb3a5aeac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:42.584870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b79ebf-0aa1-463f-80c0-9a4f51b21824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:42.585525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b79ebf-0aa1-463f-80c0-9a4f51b21824 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:42.585808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b79ebf-0aa1-463f-80c0-9a4f51b21824 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:42.586084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b79ebf-0aa1-463f-80c0-9a4f51b21824 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:42.586796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b79ebf-0aa1-463f-80c0-9a4f51b21824 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:42.588290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b79ebf-0aa1-463f-80c0-9a4f51b21824 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:42.607432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 43/85] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:27:42 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:27:42.608352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b79ebf-0aa1-463f-80c0-9a4f51b21824 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:42.628682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:42.649001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914d2e6-d5af-489e-b235-796cb3a5aeac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:42.650567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914d2e6-d5af-489e-b235-796cb3a5aeac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:42.733417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b79ebf-0aa1-463f-80c0-9a4f51b21824 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:42.764556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 43/86] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:42.765266 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914d2e6-d5af-489e-b235-796cb3a5aeac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:42.781252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade90b7-f9dd-412b-920f-ed44ece2ff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:42.782294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade90b7-f9dd-412b-920f-ed44ece2ff16 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:42.782590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade90b7-f9dd-412b-920f-ed44ece2ff16 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:42.782890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade90b7-f9dd-412b-920f-ed44ece2ff16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:42.789028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:42.867660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade90b7-f9dd-412b-920f-ed44ece2ff16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:42.869251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade90b7-f9dd-412b-920f-ed44ece2ff16 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:42.874913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914d2e6-d5af-489e-b235-796cb3a5aeac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:42.891484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:42.916356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d1040-49df-494f-a93f-2bf66ef67728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:42.917261 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d1040-49df-494f-a93f-2bf66ef67728 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:42.917550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d1040-49df-494f-a93f-2bf66ef67728 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:42.917835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d1040-49df-494f-a93f-2bf66ef67728 None admin] PATH_INFO: `/v3/users/6d22fba8da874faab7222240ee9e9e8c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:42.920288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d1040-49df-494f-a93f-2bf66ef67728 None admin] RBAC: Authorizing `identity:get_user(user_id=6d22fba8da874faab7222240ee9e9e8c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:42.922009 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d1040-49df-494f-a93f-2bf66ef67728 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:42.924546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 44/87] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:42 2022] GET /identity/v3/users/6d22fba8da874faab7222240ee9e9e8c => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:42.926103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d1040-49df-494f-a93f-2bf66ef67728 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:43.001965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 44/88] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:43.002660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade90b7-f9dd-412b-920f-ed44ece2ff16 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:43.037186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:43.061748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d1040-49df-494f-a93f-2bf66ef67728 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:43.111776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814df4d-d6d1-4a93-8f85-c683fdfedb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:43.113620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814df4d-d6d1-4a93-8f85-c683fdfedb23 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:43.114085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814df4d-d6d1-4a93-8f85-c683fdfedb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:43.114771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814df4d-d6d1-4a93-8f85-c683fdfedb23 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:43.116412 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814df4d-d6d1-4a93-8f85-c683fdfedb23 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:43.118387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814df4d-d6d1-4a93-8f85-c683fdfedb23 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:43.120863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 45/89] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:42 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:43.122215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814df4d-d6d1-4a93-8f85-c683fdfedb23 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:43.133435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade90b7-f9dd-412b-920f-ed44ece2ff16 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:43.156047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:43.177690 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5de61-86bb-432b-b496-b34891575c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:43.179026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5de61-86bb-432b-b496-b34891575c4c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:43.179381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5de61-86bb-432b-b496-b34891575c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:43.179753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5de61-86bb-432b-b496-b34891575c4c None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:43.181585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5de61-86bb-432b-b496-b34891575c4c None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:43.183387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5de61-86bb-432b-b496-b34891575c4c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:43.185976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 45/90] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:43 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:43.188210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5de61-86bb-432b-b496-b34891575c4c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:43.254432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814df4d-d6d1-4a93-8f85-c683fdfedb23 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:43.291739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2fc47-00bb-49ca-98cf-7717323fc86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:43.292502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2fc47-00bb-49ca-98cf-7717323fc86e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:43.292804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2fc47-00bb-49ca-98cf-7717323fc86e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:43.293091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2fc47-00bb-49ca-98cf-7717323fc86e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:43.305850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5de61-86bb-432b-b496-b34891575c4c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:43.319051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:43.344160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103673da-1ff3-4ab3-a5f5-55f86a6c5502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:43.345121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103673da-1ff3-4ab3-a5f5-55f86a6c5502 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:43.345455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103673da-1ff3-4ab3-a5f5-55f86a6c5502 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:43.345776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103673da-1ff3-4ab3-a5f5-55f86a6c5502 None admin] PATH_INFO: `/v3/projects/1a86ec93eeee48d8ae3158daa5e8fbf2` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:43.347144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103673da-1ff3-4ab3-a5f5-55f86a6c5502 None admin] RBAC: Authorizing `identity:get_project(project_id=1a86ec93eeee48d8ae3158daa5e8fbf2)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:43.348830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103673da-1ff3-4ab3-a5f5-55f86a6c5502 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:43.350956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 46/91] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:43 2022] GET /identity/v3/projects/1a86ec93eeee48d8ae3158daa5e8fbf2 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:43.352127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103673da-1ff3-4ab3-a5f5-55f86a6c5502 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:43.372220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2fc47-00bb-49ca-98cf-7717323fc86e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:43.373853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2fc47-00bb-49ca-98cf-7717323fc86e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:43.425802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:43.490561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103673da-1ff3-4ab3-a5f5-55f86a6c5502 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:43.491538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 46/92] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:43.492559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2fc47-00bb-49ca-98cf-7717323fc86e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:43.524897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6d8ec-82d2-4da2-bbec-e3526130c2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:43.525569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6d8ec-82d2-4da2-bbec-e3526130c2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:43.525876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6d8ec-82d2-4da2-bbec-e3526130c2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:43.526130 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6d8ec-82d2-4da2-bbec-e3526130c2f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:43.526970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6d8ec-82d2-4da2-bbec-e3526130c2f5 None admin] RBAC: Adding query filter params (role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=ab57ebe7c2f9472facbe45d50407af91, user.id=6d22fba8da874faab7222240ee9e9e8c) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:43.527307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6d8ec-82d2-4da2-bbec-e3526130c2f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:43.528791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6d8ec-82d2-4da2-bbec-e3526130c2f5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:43.534436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 47/93] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:43 2022] GET /identity/v3/role_assignments?user.id=6d22fba8da874faab7222240ee9e9e8c&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=0428a514a20043c680400ebf6498c74e => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:43.535227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6d8ec-82d2-4da2-bbec-e3526130c2f5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:43.578554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:43.643485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6d8ec-82d2-4da2-bbec-e3526130c2f5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:43.664253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:43.677905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef3dcb-dac4-49a5-9c5b-c54149445254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:43.678730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef3dcb-dac4-49a5-9c5b-c54149445254 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:43.683482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef3dcb-dac4-49a5-9c5b-c54149445254 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:43.683482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef3dcb-dac4-49a5-9c5b-c54149445254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:43.683482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef3dcb-dac4-49a5-9c5b-c54149445254 None admin] RBAC: Adding query filter params (role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=1a86ec93eeee48d8ae3158daa5e8fbf2, user.id=6d22fba8da874faab7222240ee9e9e8c) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:43.683482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef3dcb-dac4-49a5-9c5b-c54149445254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:43.683482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef3dcb-dac4-49a5-9c5b-c54149445254 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:43.689176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 48/94] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:43 2022] GET /identity/v3/role_assignments?user.id=6d22fba8da874faab7222240ee9e9e8c&scope.project.id=1a86ec93eeee48d8ae3158daa5e8fbf2&role.id=0428a514a20043c680400ebf6498c74e => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:43.690253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef3dcb-dac4-49a5-9c5b-c54149445254 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:43.725932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2fc47-00bb-49ca-98cf-7717323fc86e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:43.814918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef3dcb-dac4-49a5-9c5b-c54149445254 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:43.855082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ccdc6-6977-41da-a779-dae009bb73bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:43.856177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ccdc6-6977-41da-a779-dae009bb73bf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:43.856614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ccdc6-6977-41da-a779-dae009bb73bf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:43.857139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ccdc6-6977-41da-a779-dae009bb73bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:43.857758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b8b22-113e-4077-adbb-bb47a7b5575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:43.858537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ccdc6-6977-41da-a779-dae009bb73bf None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=ab57ebe7c2f9472facbe45d50407af91, user.id=645d7df5139643969242f8464ac102fe) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:43.858869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b8b22-113e-4077-adbb-bb47a7b5575b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:43.859231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b8b22-113e-4077-adbb-bb47a7b5575b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:43.859415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ccdc6-6977-41da-a779-dae009bb73bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:43.859747 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b8b22-113e-4077-adbb-bb47a7b5575b None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:43.861261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b8b22-113e-4077-adbb-bb47a7b5575b None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:43.861967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ccdc6-6977-41da-a779-dae009bb73bf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:43.865166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b8b22-113e-4077-adbb-bb47a7b5575b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:43.865166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 49/95] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:43 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:43.871083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b8b22-113e-4077-adbb-bb47a7b5575b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:43.893530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 47/96] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:43 2022] GET /identity/v3/role_assignments?user.id=645d7df5139643969242f8464ac102fe&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 259 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:43.978096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:44.067169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b8b22-113e-4077-adbb-bb47a7b5575b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:44.133883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48228f03-ee0e-4ae9-b078-9efea5a241e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:44.135053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48228f03-ee0e-4ae9-b078-9efea5a241e4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:44.135636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48228f03-ee0e-4ae9-b078-9efea5a241e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:44.136229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48228f03-ee0e-4ae9-b078-9efea5a241e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:44.137378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48228f03-ee0e-4ae9-b078-9efea5a241e4 None admin] RBAC: Adding query filter params (role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=ab57ebe7c2f9472facbe45d50407af91, user.id=6d22fba8da874faab7222240ee9e9e8c) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:44.137975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48228f03-ee0e-4ae9-b078-9efea5a241e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:44.139884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48228f03-ee0e-4ae9-b078-9efea5a241e4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:44.152012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 50/97] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:43 2022] GET /identity/v3/role_assignments?user.id=6d22fba8da874faab7222240ee9e9e8c&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:45.390443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1363b76-f6c6-4602-864b-610a122630a0 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:45.391498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1363b76-f6c6-4602-864b-610a122630a0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:45.391990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1363b76-f6c6-4602-864b-610a122630a0 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:45.393583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 48/98] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:45.404747 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da317fd8-015e-48a8-b03f-93db94c560cf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:45.405554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da317fd8-015e-48a8-b03f-93db94c560cf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:45.406046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da317fd8-015e-48a8-b03f-93db94c560cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:45.409126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439f389-4221-414b-9d75-b5bcac303b29 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:45.409645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439f389-4221-414b-9d75-b5bcac303b29 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:45.410118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439f389-4221-414b-9d75-b5bcac303b29 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:45.411683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 49/99] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:45.416693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1093f-59fe-4660-9523-69698834c280 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:45.417238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1093f-59fe-4660-9523-69698834c280 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:45.417709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1093f-59fe-4660-9523-69698834c280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:45.428688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-da317fd8-015e-48a8-b03f-93db94c560cf None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:45.430145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-da317fd8-015e-48a8-b03f-93db94c560cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:45.445034 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d5e1093f-59fe-4660-9523-69698834c280 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:45.446495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d5e1093f-59fe-4660-9523-69698834c280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:45.514654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 50/100] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 13 20:27:45.566952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1093f-59fe-4660-9523-69698834c280 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:45.567900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 51/101] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:45.602204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da317fd8-015e-48a8-b03f-93db94c560cf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:45.639171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:45.691184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:45.698609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1093f-59fe-4660-9523-69698834c280 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:45.734172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbea4e8-289a-4815-982f-813f64e3dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:45.735278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbea4e8-289a-4815-982f-813f64e3dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:45.735757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbea4e8-289a-4815-982f-813f64e3dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:45.736234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbea4e8-289a-4815-982f-813f64e3dc8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:45.746852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbea4e8-289a-4815-982f-813f64e3dc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4bbea4e8-289a-4815-982f-813f64e3dc8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:45.754892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:45.755935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:27:45.756755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:45.760253 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 51/102] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:45.765946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbea4e8-289a-4815-982f-813f64e3dc8a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:45.833758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da317fd8-015e-48a8-b03f-93db94c560cf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:45.880697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbea4e8-289a-4815-982f-813f64e3dc8a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:45.890556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:45.918855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b114a73-7972-4df6-9195-5860e7d070e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:45.920048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b114a73-7972-4df6-9195-5860e7d070e5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:45.920495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b114a73-7972-4df6-9195-5860e7d070e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:45.921613 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b114a73-7972-4df6-9195-5860e7d070e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:45.924189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b114a73-7972-4df6-9195-5860e7d070e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:45.925812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b114a73-7972-4df6-9195-5860e7d070e5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:45.930467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 52/103] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:45.936254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b114a73-7972-4df6-9195-5860e7d070e5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:45.941285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0143831-8f5d-4eb6-9a93-495675bfef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:45.942126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0143831-8f5d-4eb6-9a93-495675bfef39 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:45.946112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0143831-8f5d-4eb6-9a93-495675bfef39 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:45.946555 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0143831-8f5d-4eb6-9a93-495675bfef39 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:45.947484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0143831-8f5d-4eb6-9a93-495675bfef39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:45.947931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0143831-8f5d-4eb6-9a93-495675bfef39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:45.950189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0143831-8f5d-4eb6-9a93-495675bfef39 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:45.959899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 52/104] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:45.968536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0143831-8f5d-4eb6-9a93-495675bfef39 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:46.120774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0143831-8f5d-4eb6-9a93-495675bfef39 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:46.168451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469284c-7d97-45eb-a4eb-35a645e39d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:46.169538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469284c-7d97-45eb-a4eb-35a645e39d56 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.169830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469284c-7d97-45eb-a4eb-35a645e39d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.170171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469284c-7d97-45eb-a4eb-35a645e39d56 None admin] PATH_INFO: `/v3/users/74bb1fcf11c04f73a0b14c3144de90e4` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.185137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469284c-7d97-45eb-a4eb-35a645e39d56 None admin] RBAC: Authorizing `identity:get_user(user_id=74bb1fcf11c04f73a0b14c3144de90e4)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:46.186781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469284c-7d97-45eb-a4eb-35a645e39d56 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:46.189179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 53/105] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:45 2022] GET /identity/v3/users/74bb1fcf11c04f73a0b14c3144de90e4 => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:46.194732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469284c-7d97-45eb-a4eb-35a645e39d56 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:46.225499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:46.227508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b114a73-7972-4df6-9195-5860e7d070e5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:46.253044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:46.305952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469284c-7d97-45eb-a4eb-35a645e39d56 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:46.308019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c806e4-e9e4-4e67-9ce9-b03044cda60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:46.308811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c806e4-e9e4-4e67-9ce9-b03044cda60f None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.309116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c806e4-e9e4-4e67-9ce9-b03044cda60f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.309408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c806e4-e9e4-4e67-9ce9-b03044cda60f None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.310140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c806e4-e9e4-4e67-9ce9-b03044cda60f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:46.311733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c806e4-e9e4-4e67-9ce9-b03044cda60f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:46.342240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe9200-3a9d-4b21-836c-69215a7efc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:46.342909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe9200-3a9d-4b21-836c-69215a7efc4a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.343206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe9200-3a9d-4b21-836c-69215a7efc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.343511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe9200-3a9d-4b21-836c-69215a7efc4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.370097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 53/106] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:27:45 2022] POST /identity/v3/projects => generated 302 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:27:46.371697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c7416-5daf-4235-993f-abf8cfc8f022 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.371997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c7416-5daf-4235-993f-abf8cfc8f022 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.372284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c7416-5daf-4235-993f-abf8cfc8f022 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.373652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 54/107] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:46.380009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab6fab-a04a-4d24-b8d4-707a2b83ff8a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.380328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab6fab-a04a-4d24-b8d4-707a2b83ff8a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.380588 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab6fab-a04a-4d24-b8d4-707a2b83ff8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.416674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8aab6fab-a04a-4d24-b8d4-707a2b83ff8a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:46.418136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8aab6fab-a04a-4d24-b8d4-707a2b83ff8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:46.421030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe9200-3a9d-4b21-836c-69215a7efc4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:46.422624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe9200-3a9d-4b21-836c-69215a7efc4a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:46.455557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:46.585576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 55/108] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:46.610095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab6fab-a04a-4d24-b8d4-707a2b83ff8a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:46.619822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 54/109] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:46.628919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe9200-3a9d-4b21-836c-69215a7efc4a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:46.714397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:46.725729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab6fab-a04a-4d24-b8d4-707a2b83ff8a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:46.766320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec78a1-509f-4a51-bf55-c71753adc681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:46.768243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec78a1-509f-4a51-bf55-c71753adc681 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.768955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec78a1-509f-4a51-bf55-c71753adc681 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.769380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec78a1-509f-4a51-bf55-c71753adc681 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.791931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec78a1-509f-4a51-bf55-c71753adc681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-49ec78a1-509f-4a51-bf55-c71753adc681 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:46.800899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:46.801770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:46.802492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:46.805063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 56/110] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:27:46.811281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec78a1-509f-4a51-bf55-c71753adc681 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:46.837669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe9200-3a9d-4b21-836c-69215a7efc4a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:46.888488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:46.908790 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154fc9e-046a-4a14-9b13-2966338b58a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:46.909735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154fc9e-046a-4a14-9b13-2966338b58a1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.910076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154fc9e-046a-4a14-9b13-2966338b58a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.910389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154fc9e-046a-4a14-9b13-2966338b58a1 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.912168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154fc9e-046a-4a14-9b13-2966338b58a1 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:46.913667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154fc9e-046a-4a14-9b13-2966338b58a1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:46.915714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 55/111] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:46 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:46.924937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec78a1-509f-4a51-bf55-c71753adc681 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:46.929522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154fc9e-046a-4a14-9b13-2966338b58a1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:46.964933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43502c11-fc96-4f77-95a6-f6d11a0b7957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:46.965938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43502c11-fc96-4f77-95a6-f6d11a0b7957 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.966673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43502c11-fc96-4f77-95a6-f6d11a0b7957 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.968443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43502c11-fc96-4f77-95a6-f6d11a0b7957 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.969402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43502c11-fc96-4f77-95a6-f6d11a0b7957 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:46.971512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43502c11-fc96-4f77-95a6-f6d11a0b7957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:46.973144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43502c11-fc96-4f77-95a6-f6d11a0b7957 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:46.983495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 57/112] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:46.985087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a40194-058c-45d2-959c-5dfee96c26b3 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.985580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a40194-058c-45d2-959c-5dfee96c26b3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.986057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a40194-058c-45d2-959c-5dfee96c26b3 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.987771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 58/113] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:46.989139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376f30c-2f1d-471c-812f-32f64103daa7 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.989638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376f30c-2f1d-471c-812f-32f64103daa7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.990111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376f30c-2f1d-471c-812f-32f64103daa7 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:46.991554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 59/114] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:46.993012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f521c10-8912-4d25-8538-09478d80e406 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:46.993500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f521c10-8912-4d25-8538-09478d80e406 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:46.994010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f521c10-8912-4d25-8538-09478d80e406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.011213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6f521c10-8912-4d25-8538-09478d80e406 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:47.012738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6f521c10-8912-4d25-8538-09478d80e406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:47.023186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:47.061438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154fc9e-046a-4a14-9b13-2966338b58a1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:47.089001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 60/115] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:47.090165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f521c10-8912-4d25-8538-09478d80e406 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:47.108709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745c109-0ab1-4668-8a93-0b0e97377c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:47.109456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745c109-0ab1-4668-8a93-0b0e97377c4b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.109754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745c109-0ab1-4668-8a93-0b0e97377c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.110041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745c109-0ab1-4668-8a93-0b0e97377c4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.110899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745c109-0ab1-4668-8a93-0b0e97377c4b None admin] RBAC: Adding query filter params (role.id=b27dca438b9d4942b4739e98f7796831, scope.project.id=ab57ebe7c2f9472facbe45d50407af91, user.id=74bb1fcf11c04f73a0b14c3144de90e4) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:47.111274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745c109-0ab1-4668-8a93-0b0e97377c4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:47.112798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745c109-0ab1-4668-8a93-0b0e97377c4b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:47.119376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 56/116] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:46 2022] GET /identity/v3/role_assignments?user.id=74bb1fcf11c04f73a0b14c3144de90e4&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=b27dca438b9d4942b4739e98f7796831 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:47.120754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9496e0-10da-4eeb-921d-9e523b399d42 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.121055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9496e0-10da-4eeb-921d-9e523b399d42 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.121343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9496e0-10da-4eeb-921d-9e523b399d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.160710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3b9496e0-10da-4eeb-921d-9e523b399d42 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:47.161966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3b9496e0-10da-4eeb-921d-9e523b399d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:47.205290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f521c10-8912-4d25-8538-09478d80e406 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:47.243353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e74c5a-f552-41fc-83a9-9b5573f15d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:47.244502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e74c5a-f552-41fc-83a9-9b5573f15d2a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.244939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e74c5a-f552-41fc-83a9-9b5573f15d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.245384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e74c5a-f552-41fc-83a9-9b5573f15d2a None admin] PATH_INFO: `/v3/users/6d22fba8da874faab7222240ee9e9e8c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.246847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e74c5a-f552-41fc-83a9-9b5573f15d2a None admin] RBAC: Authorizing `identity:get_user(user_id=6d22fba8da874faab7222240ee9e9e8c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:47.248646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e74c5a-f552-41fc-83a9-9b5573f15d2a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:47.250910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 61/117] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:47 2022] GET /identity/v3/users/6d22fba8da874faab7222240ee9e9e8c => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:47.252009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e74c5a-f552-41fc-83a9-9b5573f15d2a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:47.255379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:47.268706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:47.271737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 57/118] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:47.276928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600537da-8a6d-4d83-be76-1705e55629ed None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.277291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600537da-8a6d-4d83-be76-1705e55629ed None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.279743 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600537da-8a6d-4d83-be76-1705e55629ed None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.281741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 58/119] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:47.282646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600537da-8a6d-4d83-be76-1705e55629ed None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:47.366633 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e74c5a-f552-41fc-83a9-9b5573f15d2a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:47.403061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713743ef-5935-4c4a-af02-18fd80d98ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:47.403939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713743ef-5935-4c4a-af02-18fd80d98ece None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.404228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713743ef-5935-4c4a-af02-18fd80d98ece None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.404591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713743ef-5935-4c4a-af02-18fd80d98ece None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.414160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600537da-8a6d-4d83-be76-1705e55629ed None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:47.415882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713743ef-5935-4c4a-af02-18fd80d98ece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-713743ef-5935-4c4a-af02-18fd80d98ece None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:47.424051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:47.425318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:47.426047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 62/120] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:47.427199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2e04c-ccbc-473e-a8c1-e29567862b97 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.427199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2e04c-ccbc-473e-a8c1-e29567862b97 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.427431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2e04c-ccbc-473e-a8c1-e29567862b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.446038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dcc2e04c-ccbc-473e-a8c1-e29567862b97 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:47.447238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dcc2e04c-ccbc-473e-a8c1-e29567862b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:47.457503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7878db-516c-437e-a0b9-d071e66b2bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:47.458808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7878db-516c-437e-a0b9-d071e66b2bfb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.459535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7878db-516c-437e-a0b9-d071e66b2bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.459832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7878db-516c-437e-a0b9-d071e66b2bfb None admin] PATH_INFO: `/v3/projects/1a86ec93eeee48d8ae3158daa5e8fbf2` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.461428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7878db-516c-437e-a0b9-d071e66b2bfb None admin] RBAC: Authorizing `identity:get_project(project_id=1a86ec93eeee48d8ae3158daa5e8fbf2)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:47.463866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7878db-516c-437e-a0b9-d071e66b2bfb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:47.466420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 59/121] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:47 2022] GET /identity/v3/projects/1a86ec93eeee48d8ae3158daa5e8fbf2 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:47.467664 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7878db-516c-437e-a0b9-d071e66b2bfb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:47.527635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 63/122] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:47.528460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2e04c-ccbc-473e-a8c1-e29567862b97 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:47.532380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:47.556448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:47.600544 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7878db-516c-437e-a0b9-d071e66b2bfb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:47.639017 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476af84a-3693-454f-84af-e756d04b8eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:47.640375 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476af84a-3693-454f-84af-e756d04b8eda None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.640673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476af84a-3693-454f-84af-e756d04b8eda None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.640957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476af84a-3693-454f-84af-e756d04b8eda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.653790 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476af84a-3693-454f-84af-e756d04b8eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-476af84a-3693-454f-84af-e756d04b8eda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:47.665008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:47.667302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 60/123] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:27:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:47.669249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476af84a-3693-454f-84af-e756d04b8eda None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:47.675521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2e04c-ccbc-473e-a8c1-e29567862b97 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:47.715178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac237e-a701-4666-b0c4-13da6d226f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:47.715943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac237e-a701-4666-b0c4-13da6d226f84 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.716233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac237e-a701-4666-b0c4-13da6d226f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.716509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac237e-a701-4666-b0c4-13da6d226f84 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.717484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac237e-a701-4666-b0c4-13da6d226f84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:47.717637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac237e-a701-4666-b0c4-13da6d226f84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:47.719039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac237e-a701-4666-b0c4-13da6d226f84 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:47.728512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 64/124] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:47.730201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac237e-a701-4666-b0c4-13da6d226f84 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:47.802286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476af84a-3693-454f-84af-e756d04b8eda None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:47.808885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:47.824947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:47.847159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44890274-63d2-4d9b-8451-85a2423cb162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:47.851681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44890274-63d2-4d9b-8451-85a2423cb162 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.852164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44890274-63d2-4d9b-8451-85a2423cb162 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.852890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44890274-63d2-4d9b-8451-85a2423cb162 None admin] PATH_INFO: `/v3/projects/1a86ec93eeee48d8ae3158daa5e8fbf2/users/6d22fba8da874faab7222240ee9e9e8c/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.855893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44890274-63d2-4d9b-8451-85a2423cb162 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a86ec93eeee48d8ae3158daa5e8fbf2, user_id=6d22fba8da874faab7222240ee9e9e8c, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:47.858036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44890274-63d2-4d9b-8451-85a2423cb162 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:47.858440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac237e-a701-4666-b0c4-13da6d226f84 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:47.876024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 61/125] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:27:47 2022] PUT /identity/v3/projects/1a86ec93eeee48d8ae3158daa5e8fbf2/users/6d22fba8da874faab7222240ee9e9e8c/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:27:47.877831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44890274-63d2-4d9b-8451-85a2423cb162 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:47.919853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82363908-8a34-430f-ad3f-64a6027b6d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:47.920728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82363908-8a34-430f-ad3f-64a6027b6d47 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:47.921026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82363908-8a34-430f-ad3f-64a6027b6d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:47.921389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82363908-8a34-430f-ad3f-64a6027b6d47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:47.964765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82363908-8a34-430f-ad3f-64a6027b6d47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-82363908-8a34-430f-ad3f-64a6027b6d47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:47.982062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:47.983074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:47.983837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 65/126] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:27:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:47.984456 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82363908-8a34-430f-ad3f-64a6027b6d47 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:48.019246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44890274-63d2-4d9b-8451-85a2423cb162 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:48.064003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d40b4-f73e-4531-9dcd-88d316f2163d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:48.064968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d40b4-f73e-4531-9dcd-88d316f2163d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:48.065792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d40b4-f73e-4531-9dcd-88d316f2163d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:48.066095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d40b4-f73e-4531-9dcd-88d316f2163d None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:48.067083 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d40b4-f73e-4531-9dcd-88d316f2163d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:48.067552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d40b4-f73e-4531-9dcd-88d316f2163d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:48.069007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d40b4-f73e-4531-9dcd-88d316f2163d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:48.080867 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 62/127] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:27:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:48.081546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d40b4-f73e-4531-9dcd-88d316f2163d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:48.134450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:48.207224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d40b4-f73e-4531-9dcd-88d316f2163d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:48.245896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2c5d0-112f-4bd2-aec0-1b7b9ef82dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:48.247007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2c5d0-112f-4bd2-aec0-1b7b9ef82dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:48.247491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2c5d0-112f-4bd2-aec0-1b7b9ef82dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:48.248169 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2c5d0-112f-4bd2-aec0-1b7b9ef82dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:48.249468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2c5d0-112f-4bd2-aec0-1b7b9ef82dc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:48.249928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2c5d0-112f-4bd2-aec0-1b7b9ef82dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:48.251667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2c5d0-112f-4bd2-aec0-1b7b9ef82dc0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:48.253958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82363908-8a34-430f-ad3f-64a6027b6d47 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:48.262265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 63/128] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:48.263347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2c5d0-112f-4bd2-aec0-1b7b9ef82dc0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:48.320052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:48.357253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ba140-6711-412b-879f-d193f7bbfe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:48.358353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ba140-6711-412b-879f-d193f7bbfe5d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:48.359023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ba140-6711-412b-879f-d193f7bbfe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:48.359128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ba140-6711-412b-879f-d193f7bbfe5d None admin] PATH_INFO: `/v3/users/6d22fba8da874faab7222240ee9e9e8c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:48.362564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ba140-6711-412b-879f-d193f7bbfe5d None admin] RBAC: Authorizing `identity:get_user(user_id=6d22fba8da874faab7222240ee9e9e8c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:48.364290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ba140-6711-412b-879f-d193f7bbfe5d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:48.366676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 66/129] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:47 2022] GET /identity/v3/users/6d22fba8da874faab7222240ee9e9e8c => generated 275 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:48.367779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ba140-6711-412b-879f-d193f7bbfe5d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:48.386143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2c5d0-112f-4bd2-aec0-1b7b9ef82dc0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:48.388054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:48.446930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a634b-72a8-4062-86fb-74d5d15cc226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:48.447863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a634b-72a8-4062-86fb-74d5d15cc226 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:48.448154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a634b-72a8-4062-86fb-74d5d15cc226 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:48.448435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a634b-72a8-4062-86fb-74d5d15cc226 None admin] PATH_INFO: `/v3/users/6d22fba8da874faab7222240ee9e9e8c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:48.449976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a634b-72a8-4062-86fb-74d5d15cc226 None admin] RBAC: Authorizing `identity:get_user(user_id=6d22fba8da874faab7222240ee9e9e8c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:48.451611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a634b-72a8-4062-86fb-74d5d15cc226 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:48.455283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 64/130] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:48 2022] GET /identity/v3/users/6d22fba8da874faab7222240ee9e9e8c => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:48.456097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a634b-72a8-4062-86fb-74d5d15cc226 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:48.553114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ba140-6711-412b-879f-d193f7bbfe5d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:48.612982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c73d5f-34c5-42fb-87d5-aa582e86be20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:48.613799 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c73d5f-34c5-42fb-87d5-aa582e86be20 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:48.614087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c73d5f-34c5-42fb-87d5-aa582e86be20 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:48.614369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c73d5f-34c5-42fb-87d5-aa582e86be20 None admin] PATH_INFO: `/v3/users/645d7df5139643969242f8464ac102fe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:48.615593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c73d5f-34c5-42fb-87d5-aa582e86be20 None admin] RBAC: Authorizing `identity:get_user(user_id=645d7df5139643969242f8464ac102fe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:48.617075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c73d5f-34c5-42fb-87d5-aa582e86be20 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:48.618719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 67/131] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:48 2022] GET /identity/v3/users/645d7df5139643969242f8464ac102fe => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:48.620576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c73d5f-34c5-42fb-87d5-aa582e86be20 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:48.638720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a634b-72a8-4062-86fb-74d5d15cc226 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:48.702821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca27a7-1a7a-4433-926b-03e135f43c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:48.703741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca27a7-1a7a-4433-926b-03e135f43c3b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:48.704034 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca27a7-1a7a-4433-926b-03e135f43c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:48.704315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca27a7-1a7a-4433-926b-03e135f43c3b None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:48.706146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca27a7-1a7a-4433-926b-03e135f43c3b None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:48.707670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca27a7-1a7a-4433-926b-03e135f43c3b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:48.709387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 65/132] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:48 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:48.710381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca27a7-1a7a-4433-926b-03e135f43c3b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:48.736671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:48.787613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:48.829385 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c73d5f-34c5-42fb-87d5-aa582e86be20 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:48.856026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca27a7-1a7a-4433-926b-03e135f43c3b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:48.893498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dadab5-1c8a-42e8-b9d3-927132718bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:48.894434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dadab5-1c8a-42e8-b9d3-927132718bcf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:48.894725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dadab5-1c8a-42e8-b9d3-927132718bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:48.895009 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dadab5-1c8a-42e8-b9d3-927132718bcf None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:48.896512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dadab5-1c8a-42e8-b9d3-927132718bcf None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:48.898451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dadab5-1c8a-42e8-b9d3-927132718bcf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:48.900355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 66/133] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:48 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:48.901243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dadab5-1c8a-42e8-b9d3-927132718bcf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:48.932179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00f495-43e2-4b3d-89dd-22e0aec3f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:48.933428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00f495-43e2-4b3d-89dd-22e0aec3f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:48.933921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00f495-43e2-4b3d-89dd-22e0aec3f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:48.934417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00f495-43e2-4b3d-89dd-22e0aec3f6a7 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:48.940193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00f495-43e2-4b3d-89dd-22e0aec3f6a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:48.941932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00f495-43e2-4b3d-89dd-22e0aec3f6a7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:48.943958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 68/134] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:48 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:48.944948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00f495-43e2-4b3d-89dd-22e0aec3f6a7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:49.008681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dadab5-1c8a-42e8-b9d3-927132718bcf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:49.023804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:49.056347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df895b-b0b9-467d-8645-609ceac28f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:49.057265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df895b-b0b9-467d-8645-609ceac28f80 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.057558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df895b-b0b9-467d-8645-609ceac28f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.057837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df895b-b0b9-467d-8645-609ceac28f80 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91/users/6d22fba8da874faab7222240ee9e9e8c/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.063395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df895b-b0b9-467d-8645-609ceac28f80 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab57ebe7c2f9472facbe45d50407af91, user_id=6d22fba8da874faab7222240ee9e9e8c, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:49.065255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df895b-b0b9-467d-8645-609ceac28f80 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:49.085903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 67/135] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:27:48 2022] PUT /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91/users/6d22fba8da874faab7222240ee9e9e8c/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:27:49.086480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df895b-b0b9-467d-8645-609ceac28f80 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:49.123543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00f495-43e2-4b3d-89dd-22e0aec3f6a7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:49.220692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:49.227606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df895b-b0b9-467d-8645-609ceac28f80 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:49.252269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbf634-95f0-47a4-a682-bc6ee9ab064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:49.253197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbf634-95f0-47a4-a682-bc6ee9ab064b None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.253574 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbf634-95f0-47a4-a682-bc6ee9ab064b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.253754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbf634-95f0-47a4-a682-bc6ee9ab064b None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91/users/645d7df5139643969242f8464ac102fe/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.256536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbf634-95f0-47a4-a682-bc6ee9ab064b None admin] RBAC: Authorizing `identity:create_grant(project_id=ab57ebe7c2f9472facbe45d50407af91, user_id=645d7df5139643969242f8464ac102fe, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:49.258002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbf634-95f0-47a4-a682-bc6ee9ab064b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:49.273569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942a186-76c2-4ec0-b0e9-a1892929b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:49.275356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942a186-76c2-4ec0-b0e9-a1892929b36b None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.278197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942a186-76c2-4ec0-b0e9-a1892929b36b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.278815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942a186-76c2-4ec0-b0e9-a1892929b36b None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91/users/6d22fba8da874faab7222240ee9e9e8c/roles/32bea78dbfe74b56bd5c1f113586c690` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.285295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942a186-76c2-4ec0-b0e9-a1892929b36b None admin] RBAC: Authorizing `identity:create_grant(project_id=ab57ebe7c2f9472facbe45d50407af91, user_id=6d22fba8da874faab7222240ee9e9e8c, role_id=32bea78dbfe74b56bd5c1f113586c690)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:49.288774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942a186-76c2-4ec0-b0e9-a1892929b36b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:49.289692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 69/136] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:27:48 2022] PUT /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91/users/645d7df5139643969242f8464ac102fe/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:27:49.294165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81ec4f-ebc6-4c98-9b87-88f11eccd9bc None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.294727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81ec4f-ebc6-4c98-9b87-88f11eccd9bc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.295327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81ec4f-ebc6-4c98-9b87-88f11eccd9bc None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.299548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 70/137] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:49.306172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b567-6c43-448b-a6ee-9b447815aeab None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.306794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b567-6c43-448b-a6ee-9b447815aeab None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.307754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b567-6c43-448b-a6ee-9b447815aeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.311816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 68/138] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:27:49 2022] PUT /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91/users/6d22fba8da874faab7222240ee9e9e8c/roles/32bea78dbfe74b56bd5c1f113586c690 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:27:49.332013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fac0b567-6c43-448b-a6ee-9b447815aeab None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:49.333978 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fac0b567-6c43-448b-a6ee-9b447815aeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:49.397001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566de7b-9f7a-4af7-9bd2-2f0186940cb0 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.397334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566de7b-9f7a-4af7-9bd2-2f0186940cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.397580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566de7b-9f7a-4af7-9bd2-2f0186940cb0 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.399018 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 69/139] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:27:49.405183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78c517-2435-4f56-a7e9-ca6496507bf0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.405477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78c517-2435-4f56-a7e9-ca6496507bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.405762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78c517-2435-4f56-a7e9-ca6496507bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.425447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 71/140] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:49.452019 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-aa78c517-2435-4f56-a7e9-ca6496507bf0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:49.452580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b567-6c43-448b-a6ee-9b447815aeab None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:49.453261 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-aa78c517-2435-4f56-a7e9-ca6496507bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:49.487694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:49.577981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 70/141] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 13 20:27:49.603642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b567-6c43-448b-a6ee-9b447815aeab None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:49.618319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78c517-2435-4f56-a7e9-ca6496507bf0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:49.639505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:49.677464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e035a-d071-4b76-a4dd-7854e912d335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:49.678045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e035a-d071-4b76-a4dd-7854e912d335 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.678341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e035a-d071-4b76-a4dd-7854e912d335 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.678630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e035a-d071-4b76-a4dd-7854e912d335 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.680154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e035a-d071-4b76-a4dd-7854e912d335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:49.681882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e035a-d071-4b76-a4dd-7854e912d335 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:49.683621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 72/142] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:49.689266 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e035a-d071-4b76-a4dd-7854e912d335 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:49.759524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78c517-2435-4f56-a7e9-ca6496507bf0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:49.797183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf235e51-5c7c-4aaa-b3ec-c32f5d6ddc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:49.797999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf235e51-5c7c-4aaa-b3ec-c32f5d6ddc83 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.798302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf235e51-5c7c-4aaa-b3ec-c32f5d6ddc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.798568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf235e51-5c7c-4aaa-b3ec-c32f5d6ddc83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.809279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf235e51-5c7c-4aaa-b3ec-c32f5d6ddc83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bf235e51-5c7c-4aaa-b3ec-c32f5d6ddc83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:49.821255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:49.822170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:49.822942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 71/143] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:27:49.832471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf235e51-5c7c-4aaa-b3ec-c32f5d6ddc83 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:49.860027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e035a-d071-4b76-a4dd-7854e912d335 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:49.864208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:49.927703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aad40-a202-4687-845d-283506191b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:49.928784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aad40-a202-4687-845d-283506191b1d None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.929289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aad40-a202-4687-845d-283506191b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.929837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aad40-a202-4687-845d-283506191b1d None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.930861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1aad40-a202-4687-845d-283506191b1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:49.932601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1aad40-a202-4687-845d-283506191b1d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:49.950085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf235e51-5c7c-4aaa-b3ec-c32f5d6ddc83 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:49.970581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 73/144] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:27:49 2022] POST /identity/v3/users => generated 283 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:27:49.985126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882271-0df8-49d7-8595-f8d156943ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:49.985798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882271-0df8-49d7-8595-f8d156943ccc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:49.986080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882271-0df8-49d7-8595-f8d156943ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:49.986356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882271-0df8-49d7-8595-f8d156943ccc None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:49.987124 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33882271-0df8-49d7-8595-f8d156943ccc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:49.987419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33882271-0df8-49d7-8595-f8d156943ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:49.988865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33882271-0df8-49d7-8595-f8d156943ccc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:49.997806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 72/145] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:50.003610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aad40-a202-4687-845d-283506191b1d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:50.064851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:50.145757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aad40-a202-4687-845d-283506191b1d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:50.197460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26523e28-7913-48e1-80a1-9294f9464967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:50.198463 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26523e28-7913-48e1-80a1-9294f9464967 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:50.198812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26523e28-7913-48e1-80a1-9294f9464967 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:50.199476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26523e28-7913-48e1-80a1-9294f9464967 None admin] PATH_INFO: `/v3/users/74bb1fcf11c04f73a0b14c3144de90e4` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:50.200841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26523e28-7913-48e1-80a1-9294f9464967 None admin] RBAC: Authorizing `identity:get_user(user_id=74bb1fcf11c04f73a0b14c3144de90e4)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:50.202721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26523e28-7913-48e1-80a1-9294f9464967 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:50.204560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 74/146] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:50 2022] GET /identity/v3/users/74bb1fcf11c04f73a0b14c3144de90e4 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:50.210307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882271-0df8-49d7-8595-f8d156943ccc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:50.296405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:50.316407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882271-0df8-49d7-8595-f8d156943ccc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:50.350841 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549dfcf-deab-4235-9736-a1a4f4763551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:50.351833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549dfcf-deab-4235-9736-a1a4f4763551 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:50.352185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549dfcf-deab-4235-9736-a1a4f4763551 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:50.352515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549dfcf-deab-4235-9736-a1a4f4763551 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:50.353780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549dfcf-deab-4235-9736-a1a4f4763551 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:50.355314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549dfcf-deab-4235-9736-a1a4f4763551 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:50.357116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 73/147] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:50 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:50.363049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26523e28-7913-48e1-80a1-9294f9464967 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:50.443484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031b048-0070-44d1-9435-f4e2c99d9a9b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:50.443484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031b048-0070-44d1-9435-f4e2c99d9a9b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:50.443484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031b048-0070-44d1-9435-f4e2c99d9a9b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:50.444705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 74/148] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:50.450140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b19b5f-a0b2-4be8-85b4-cbb50afd2eee None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:50.450632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b19b5f-a0b2-4be8-85b4-cbb50afd2eee None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:50.451047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b19b5f-a0b2-4be8-85b4-cbb50afd2eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:50.474468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d8b19b5f-a0b2-4be8-85b4-cbb50afd2eee None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:50.475870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d8b19b5f-a0b2-4be8-85b4-cbb50afd2eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:50.489894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26523e28-7913-48e1-80a1-9294f9464967 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:50.501796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:50.528439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45105338-b450-4e3b-a2a1-320b6276e535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:50.529368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45105338-b450-4e3b-a2a1-320b6276e535 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:50.529664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45105338-b450-4e3b-a2a1-320b6276e535 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:50.529950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45105338-b450-4e3b-a2a1-320b6276e535 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91/users/74bb1fcf11c04f73a0b14c3144de90e4/roles/b27dca438b9d4942b4739e98f7796831` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:50.534790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45105338-b450-4e3b-a2a1-320b6276e535 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab57ebe7c2f9472facbe45d50407af91, user_id=74bb1fcf11c04f73a0b14c3144de90e4, role_id=b27dca438b9d4942b4739e98f7796831)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:50.536480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45105338-b450-4e3b-a2a1-320b6276e535 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:50.553247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 75/149] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:27:50 2022] PUT /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91/users/74bb1fcf11c04f73a0b14c3144de90e4/roles/b27dca438b9d4942b4739e98f7796831 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:27:50.571748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 75/150] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:50.595195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45105338-b450-4e3b-a2a1-320b6276e535 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:50.719166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45105338-b450-4e3b-a2a1-320b6276e535 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:50.761003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019f32f-e655-4fca-b59c-1a33bede2c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:50.762054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019f32f-e655-4fca-b59c-1a33bede2c52 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:50.762673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019f32f-e655-4fca-b59c-1a33bede2c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:50.762961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019f32f-e655-4fca-b59c-1a33bede2c52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:50.774690 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019f32f-e655-4fca-b59c-1a33bede2c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1019f32f-e655-4fca-b59c-1a33bede2c52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:50.784611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:50.785477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:50.786217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 76/151] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:50.792879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b19b5f-a0b2-4be8-85b4-cbb50afd2eee None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:50.911931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:50.933073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b19b5f-a0b2-4be8-85b4-cbb50afd2eee None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:50.971646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673cbe0-9ff2-439d-b72e-4eb25aa578a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:50.972648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673cbe0-9ff2-439d-b72e-4eb25aa578a9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:50.973114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673cbe0-9ff2-439d-b72e-4eb25aa578a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:50.973562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673cbe0-9ff2-439d-b72e-4eb25aa578a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:50.974496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673cbe0-9ff2-439d-b72e-4eb25aa578a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:50.974979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673cbe0-9ff2-439d-b72e-4eb25aa578a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:50.976593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673cbe0-9ff2-439d-b72e-4eb25aa578a9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:50.991320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 76/152] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:50.997304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019f32f-e655-4fca-b59c-1a33bede2c52 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:51.037790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:51.138183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019f32f-e655-4fca-b59c-1a33bede2c52 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:51.200822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8477f58-6084-4185-9304-49f0b2d9bc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:51.201685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8477f58-6084-4185-9304-49f0b2d9bc32 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:51.201971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8477f58-6084-4185-9304-49f0b2d9bc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:51.202255 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8477f58-6084-4185-9304-49f0b2d9bc32 None admin] PATH_INFO: `/v3/users/6d22fba8da874faab7222240ee9e9e8c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:51.203521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8477f58-6084-4185-9304-49f0b2d9bc32 None admin] RBAC: Authorizing `identity:get_user(user_id=6d22fba8da874faab7222240ee9e9e8c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:51.205001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8477f58-6084-4185-9304-49f0b2d9bc32 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:51.206877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 77/153] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:50 2022] GET /identity/v3/users/6d22fba8da874faab7222240ee9e9e8c => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:51.212187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673cbe0-9ff2-439d-b72e-4eb25aa578a9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:51.386310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673cbe0-9ff2-439d-b72e-4eb25aa578a9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:51.429666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a113e9-82fe-41dd-8042-ef45f58debe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:51.430334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a113e9-82fe-41dd-8042-ef45f58debe3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:51.430624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a113e9-82fe-41dd-8042-ef45f58debe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:51.430906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a113e9-82fe-41dd-8042-ef45f58debe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:51.444459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:51.511114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a113e9-82fe-41dd-8042-ef45f58debe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:51.512663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a113e9-82fe-41dd-8042-ef45f58debe3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:51.633114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 77/154] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:51.638781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8477f58-6084-4185-9304-49f0b2d9bc32 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:51.744001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:51.747977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8477f58-6084-4185-9304-49f0b2d9bc32 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:51.797498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111210c-91d4-40da-a5ec-97b04dc56a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:51.798307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111210c-91d4-40da-a5ec-97b04dc56a4e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:51.798595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111210c-91d4-40da-a5ec-97b04dc56a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:51.798873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111210c-91d4-40da-a5ec-97b04dc56a4e None admin] PATH_INFO: `/v3/projects/1a86ec93eeee48d8ae3158daa5e8fbf2` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:51.804192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111210c-91d4-40da-a5ec-97b04dc56a4e None admin] RBAC: Authorizing `identity:get_project(project_id=1a86ec93eeee48d8ae3158daa5e8fbf2)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:51.805673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111210c-91d4-40da-a5ec-97b04dc56a4e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:51.807424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 78/155] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:51 2022] GET /identity/v3/projects/1a86ec93eeee48d8ae3158daa5e8fbf2 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:51.813552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a113e9-82fe-41dd-8042-ef45f58debe3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:51.862111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:51.925530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a113e9-82fe-41dd-8042-ef45f58debe3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:51.962766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26836fa1-2936-44bb-8f12-0601f86b2b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:51.963441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26836fa1-2936-44bb-8f12-0601f86b2b00 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:51.963750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26836fa1-2936-44bb-8f12-0601f86b2b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:51.964026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26836fa1-2936-44bb-8f12-0601f86b2b00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:51.965026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26836fa1-2936-44bb-8f12-0601f86b2b00 None admin] RBAC: Adding query filter params (role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=1a86ec93eeee48d8ae3158daa5e8fbf2, user.id=6d22fba8da874faab7222240ee9e9e8c) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:51.965344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26836fa1-2936-44bb-8f12-0601f86b2b00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:51.966956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26836fa1-2936-44bb-8f12-0601f86b2b00 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:51.976366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 78/156] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:51 2022] GET /identity/v3/role_assignments?user.id=6d22fba8da874faab7222240ee9e9e8c&scope.project.id=1a86ec93eeee48d8ae3158daa5e8fbf2&role.id=0428a514a20043c680400ebf6498c74e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:52.464901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3a141-2b2a-4ec3-9913-f3a6e19979b2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:52.465307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3a141-2b2a-4ec3-9913-f3a6e19979b2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:52.465487 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3a141-2b2a-4ec3-9913-f3a6e19979b2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:52.466949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 79/157] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:52.473631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3abdf-9649-4fcb-9cdc-b80dba5dc22a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:52.473932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3abdf-9649-4fcb-9cdc-b80dba5dc22a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:52.474189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3abdf-9649-4fcb-9cdc-b80dba5dc22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:52.496317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3cf3abdf-9649-4fcb-9cdc-b80dba5dc22a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:52.497507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3cf3abdf-9649-4fcb-9cdc-b80dba5dc22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:52.586280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 79/158] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:52.629887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3a141-2b2a-4ec3-9913-f3a6e19979b2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:52.761341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3a141-2b2a-4ec3-9913-f3a6e19979b2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:52.802446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529db1e-595c-4238-8fcd-73a6bfac4d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:52.803268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529db1e-595c-4238-8fcd-73a6bfac4d9f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:52.803583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529db1e-595c-4238-8fcd-73a6bfac4d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:52.803869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529db1e-595c-4238-8fcd-73a6bfac4d9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:52.821194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529db1e-595c-4238-8fcd-73a6bfac4d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7529db1e-595c-4238-8fcd-73a6bfac4d9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:52.829464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:52.830396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:52.831135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:52.831757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b0cbd-7a15-43a6-b296-643ad383bad1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:52.831757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b0cbd-7a15-43a6-b296-643ad383bad1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:52.831757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b0cbd-7a15-43a6-b296-643ad383bad1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:52.832253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 80/159] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:52.832318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 80/160] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:27:52.836802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529db1e-595c-4238-8fcd-73a6bfac4d9f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:52.838508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cb0eb-faa0-4b35-a738-bbf7f96ab68b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:52.838798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cb0eb-faa0-4b35-a738-bbf7f96ab68b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:52.839382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cb0eb-faa0-4b35-a738-bbf7f96ab68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:52.875360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-108cb0eb-faa0-4b35-a738-bbf7f96ab68b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:52.876585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-108cb0eb-faa0-4b35-a738-bbf7f96ab68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:52.881143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:52.941153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:53.032720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529db1e-595c-4238-8fcd-73a6bfac4d9f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:53.037053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 81/161] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:27:53.087288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08e5a1-d1f0-4633-a5d7-8cf965313ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:53.088016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08e5a1-d1f0-4633-a5d7-8cf965313ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:53.088306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08e5a1-d1f0-4633-a5d7-8cf965313ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:53.088643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08e5a1-d1f0-4633-a5d7-8cf965313ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:53.089461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08e5a1-d1f0-4633-a5d7-8cf965313ae6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:53.089775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08e5a1-d1f0-4633-a5d7-8cf965313ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:53.091208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08e5a1-d1f0-4633-a5d7-8cf965313ae6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:53.105540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 81/162] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:53.111203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cb0eb-faa0-4b35-a738-bbf7f96ab68b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:53.118742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08e5a1-d1f0-4633-a5d7-8cf965313ae6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:53.267506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08e5a1-d1f0-4633-a5d7-8cf965313ae6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:53.292045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:53.298354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cb0eb-faa0-4b35-a738-bbf7f96ab68b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:53.317845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466f2c0-5370-4e1c-bb8a-936a1819dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:53.318668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466f2c0-5370-4e1c-bb8a-936a1819dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:53.318953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466f2c0-5370-4e1c-bb8a-936a1819dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:53.319229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466f2c0-5370-4e1c-bb8a-936a1819dbb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:53.333978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd72948-e37a-4dd7-b5a9-fc1b409fe221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:53.334997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72948-e37a-4dd7-b5a9-fc1b409fe221 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:53.335581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72948-e37a-4dd7-b5a9-fc1b409fe221 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:53.336069 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72948-e37a-4dd7-b5a9-fc1b409fe221 None admin] PATH_INFO: `/v3/users/6d22fba8da874faab7222240ee9e9e8c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:53.337359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd72948-e37a-4dd7-b5a9-fc1b409fe221 None admin] RBAC: Authorizing `identity:get_user(user_id=6d22fba8da874faab7222240ee9e9e8c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:53.338930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd72948-e37a-4dd7-b5a9-fc1b409fe221 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:53.341086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 82/163] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:53 2022] GET /identity/v3/users/6d22fba8da874faab7222240ee9e9e8c => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:53.341204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466f2c0-5370-4e1c-bb8a-936a1819dbb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:53.348410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd72948-e37a-4dd7-b5a9-fc1b409fe221 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9466f2c0-5370-4e1c-bb8a-936a1819dbb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:53.361425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:53.362320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:53.363067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 82/164] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:27:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:27:53.369870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466f2c0-5370-4e1c-bb8a-936a1819dbb1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:53.408047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:53.470180 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd72948-e37a-4dd7-b5a9-fc1b409fe221 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:53.519890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48b79f-b609-4667-80a3-cc988c6b3c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:53.520602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48b79f-b609-4667-80a3-cc988c6b3c76 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:53.520945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48b79f-b609-4667-80a3-cc988c6b3c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:53.521236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48b79f-b609-4667-80a3-cc988c6b3c76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:53.585304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466f2c0-5370-4e1c-bb8a-936a1819dbb1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:53.592879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48b79f-b609-4667-80a3-cc988c6b3c76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:53.594387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48b79f-b609-4667-80a3-cc988c6b3c76 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:53.628248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:53.673157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2f9e5-9894-468d-a1e6-4456db77f096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:53.673864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2f9e5-9894-468d-a1e6-4456db77f096 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:53.674151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2f9e5-9894-468d-a1e6-4456db77f096 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:53.674434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2f9e5-9894-468d-a1e6-4456db77f096 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:53.675197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2f9e5-9894-468d-a1e6-4456db77f096 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:53.675532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2f9e5-9894-468d-a1e6-4456db77f096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:53.676960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2f9e5-9894-468d-a1e6-4456db77f096 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:53.698035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 83/165] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:53.699826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af33c71-15f2-4f39-994f-cf43a06333a9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:53.700119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af33c71-15f2-4f39-994f-cf43a06333a9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:53.700401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af33c71-15f2-4f39-994f-cf43a06333a9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:53.701751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 84/166] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:53.708402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af33c71-15f2-4f39-994f-cf43a06333a9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:53.710129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 83/167] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:53.711515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a81f3-d8b9-42dc-8352-7114cf4030c4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:53.711813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a81f3-d8b9-42dc-8352-7114cf4030c4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:53.712093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a81f3-d8b9-42dc-8352-7114cf4030c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:53.728796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-857a81f3-d8b9-42dc-8352-7114cf4030c4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:53.729993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-857a81f3-d8b9-42dc-8352-7114cf4030c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:53.731016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:53.807803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 84/168] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:53.808641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a81f3-d8b9-42dc-8352-7114cf4030c4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:53.880027 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:53.924741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a81f3-d8b9-42dc-8352-7114cf4030c4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:53.953648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af33c71-15f2-4f39-994f-cf43a06333a9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:53.962480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67037e-2a68-4574-a3e6-5052fe3d9898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:53.963372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67037e-2a68-4574-a3e6-5052fe3d9898 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:53.963704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67037e-2a68-4574-a3e6-5052fe3d9898 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:53.963987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67037e-2a68-4574-a3e6-5052fe3d9898 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:53.965178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67037e-2a68-4574-a3e6-5052fe3d9898 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:53.966839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67037e-2a68-4574-a3e6-5052fe3d9898 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:53.968642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 85/169] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:53 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:53.969417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67037e-2a68-4574-a3e6-5052fe3d9898 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:54.021051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405acc2-317d-4feb-85bb-13e98d359c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:54.021950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405acc2-317d-4feb-85bb-13e98d359c3e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.022235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405acc2-317d-4feb-85bb-13e98d359c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.022513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405acc2-317d-4feb-85bb-13e98d359c3e None admin] PATH_INFO: `/v3/users/645d7df5139643969242f8464ac102fe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.023740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405acc2-317d-4feb-85bb-13e98d359c3e None admin] RBAC: Authorizing `identity:get_user(user_id=645d7df5139643969242f8464ac102fe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:54.025215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405acc2-317d-4feb-85bb-13e98d359c3e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:54.026871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 85/170] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:53 2022] GET /identity/v3/users/645d7df5139643969242f8464ac102fe => generated 247 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:54.035748 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405acc2-317d-4feb-85bb-13e98d359c3e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:54.114067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67037e-2a68-4574-a3e6-5052fe3d9898 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:54.163953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:54.173253 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292126f4-b476-40c3-926e-c348b4cdf13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:54.174466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292126f4-b476-40c3-926e-c348b4cdf13a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.174966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292126f4-b476-40c3-926e-c348b4cdf13a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.175390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292126f4-b476-40c3-926e-c348b4cdf13a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.193793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292126f4-b476-40c3-926e-c348b4cdf13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-292126f4-b476-40c3-926e-c348b4cdf13a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:54.208452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:54.209364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 86/171] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:27:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:54.210674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292126f4-b476-40c3-926e-c348b4cdf13a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:54.242981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:54.267141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405acc2-317d-4feb-85bb-13e98d359c3e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:54.306777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf56702-ef6e-47e4-93e4-fc99f5c370f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:54.307720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56702-ef6e-47e4-93e4-fc99f5c370f8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.308179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56702-ef6e-47e4-93e4-fc99f5c370f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.308755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56702-ef6e-47e4-93e4-fc99f5c370f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.310981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf56702-ef6e-47e4-93e4-fc99f5c370f8 None admin] RBAC: Adding query filter params (role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=ab57ebe7c2f9472facbe45d50407af91, user.id=6d22fba8da874faab7222240ee9e9e8c) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:54.311486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf56702-ef6e-47e4-93e4-fc99f5c370f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:54.313867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf56702-ef6e-47e4-93e4-fc99f5c370f8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:54.323304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 86/172] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:54 2022] GET /identity/v3/role_assignments?user.id=6d22fba8da874faab7222240ee9e9e8c&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=0428a514a20043c680400ebf6498c74e => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:54.324623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf56702-ef6e-47e4-93e4-fc99f5c370f8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:54.445205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292126f4-b476-40c3-926e-c348b4cdf13a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:54.519177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220abf4f-c8dd-4aaa-b76b-12af6aa954b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:54.520020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220abf4f-c8dd-4aaa-b76b-12af6aa954b0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.520283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220abf4f-c8dd-4aaa-b76b-12af6aa954b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.520571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220abf4f-c8dd-4aaa-b76b-12af6aa954b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.534673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf56702-ef6e-47e4-93e4-fc99f5c370f8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:54.573009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226c0bb-3b8b-4755-9519-606e9b983f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:54.574382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226c0bb-3b8b-4755-9519-606e9b983f50 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.574894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226c0bb-3b8b-4755-9519-606e9b983f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.575377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226c0bb-3b8b-4755-9519-606e9b983f50 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.576447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226c0bb-3b8b-4755-9519-606e9b983f50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:54.576969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226c0bb-3b8b-4755-9519-606e9b983f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:54.578563 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226c0bb-3b8b-4755-9519-606e9b983f50 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:54.581843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:54.587770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 87/173] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:27:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:54.589841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1ee5-081c-421c-a97d-7cd0c680e73f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.590340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1ee5-081c-421c-a97d-7cd0c680e73f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.590776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1ee5-081c-421c-a97d-7cd0c680e73f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.592415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 88/174] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:54.593767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652eafd-76f6-47aa-837d-5d5f36bc43df None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.594334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652eafd-76f6-47aa-837d-5d5f36bc43df None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.594842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652eafd-76f6-47aa-837d-5d5f36bc43df None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.596383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 89/175] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:54.597296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652eafd-76f6-47aa-837d-5d5f36bc43df None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:54.601413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:54.603745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220abf4f-c8dd-4aaa-b76b-12af6aa954b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:54.605450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220abf4f-c8dd-4aaa-b76b-12af6aa954b0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:54.725048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 87/176] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:54.726706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e15aa2-65b4-4f04-8599-894657d3fa18 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.727200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e15aa2-65b4-4f04-8599-894657d3fa18 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.727724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e15aa2-65b4-4f04-8599-894657d3fa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.728023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652eafd-76f6-47aa-837d-5d5f36bc43df None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:54.747341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b3e15aa2-65b4-4f04-8599-894657d3fa18 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:54.749103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b3e15aa2-65b4-4f04-8599-894657d3fa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:54.770716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1645882-f5e1-49c3-a753-4fba17186b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:54.771638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1645882-f5e1-49c3-a753-4fba17186b01 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.771929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1645882-f5e1-49c3-a753-4fba17186b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.772212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1645882-f5e1-49c3-a753-4fba17186b01 None admin] PATH_INFO: `/v3/users/6d22fba8da874faab7222240ee9e9e8c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.773427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1645882-f5e1-49c3-a753-4fba17186b01 None admin] RBAC: Authorizing `identity:get_user(user_id=6d22fba8da874faab7222240ee9e9e8c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:54.775350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1645882-f5e1-49c3-a753-4fba17186b01 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:54.777517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 90/177] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:54 2022] GET /identity/v3/users/6d22fba8da874faab7222240ee9e9e8c => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:54.782286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5a18c-d8b8-4c3b-a6f9-9b8e7220a5da None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.782286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5a18c-d8b8-4c3b-a6f9-9b8e7220a5da None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.782286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5a18c-d8b8-4c3b-a6f9-9b8e7220a5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.799740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2ac5a18c-d8b8-4c3b-a6f9-9b8e7220a5da None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:54.800945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2ac5a18c-d8b8-4c3b-a6f9-9b8e7220a5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:54.832236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 88/178] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:54.833685 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e15aa2-65b4-4f04-8599-894657d3fa18 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:54.883924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:54.886362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 91/179] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:54.887376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5a18c-d8b8-4c3b-a6f9-9b8e7220a5da None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:54.889986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:54.943923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e15aa2-65b4-4f04-8599-894657d3fa18 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:54.982749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417e392-15e9-4ada-a0bc-32ac36467d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:54.983571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417e392-15e9-4ada-a0bc-32ac36467d8d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:54.983859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417e392-15e9-4ada-a0bc-32ac36467d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:54.984137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417e392-15e9-4ada-a0bc-32ac36467d8d None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:54.985364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417e392-15e9-4ada-a0bc-32ac36467d8d None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:54.986859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417e392-15e9-4ada-a0bc-32ac36467d8d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:54.988904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 89/180] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:54 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:54.989713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417e392-15e9-4ada-a0bc-32ac36467d8d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.000608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5a18c-d8b8-4c3b-a6f9-9b8e7220a5da None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.035872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d1b31-389a-453c-9498-b9345dc52100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.036614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d1b31-389a-453c-9498-b9345dc52100 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.036903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d1b31-389a-453c-9498-b9345dc52100 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.037181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d1b31-389a-453c-9498-b9345dc52100 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.080085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:55.095977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417e392-15e9-4ada-a0bc-32ac36467d8d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.107410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d1b31-389a-453c-9498-b9345dc52100 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:55.108912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d1b31-389a-453c-9498-b9345dc52100 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:55.110836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:55.128882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87206-5a89-4a72-8913-0b674a27b7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.129690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87206-5a89-4a72-8913-0b674a27b7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.129979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87206-5a89-4a72-8913-0b674a27b7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.130480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87206-5a89-4a72-8913-0b674a27b7a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.140602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b87206-5a89-4a72-8913-0b674a27b7a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-22b87206-5a89-4a72-8913-0b674a27b7a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:55.148378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:55.149249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:55.150141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 90/181] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:55.152626 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd6888-35e3-4c8d-9dcb-06ad0bd1dd9a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.153076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd6888-35e3-4c8d-9dcb-06ad0bd1dd9a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.153359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd6888-35e3-4c8d-9dcb-06ad0bd1dd9a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.154755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 91/182] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:55.155521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd6888-35e3-4c8d-9dcb-06ad0bd1dd9a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.206234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 92/183] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:55.207262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d1b31-389a-453c-9498-b9345dc52100 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.258454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd6888-35e3-4c8d-9dcb-06ad0bd1dd9a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.295239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1815c9-d403-458c-8b73-265af0280724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.296059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1815c9-d403-458c-8b73-265af0280724 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.296345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1815c9-d403-458c-8b73-265af0280724 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.296620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1815c9-d403-458c-8b73-265af0280724 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.303441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d1b31-389a-453c-9498-b9345dc52100 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.304973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:55.306493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1815c9-d403-458c-8b73-265af0280724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1c1815c9-d403-458c-8b73-265af0280724 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:55.314228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:55.315095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 92/184] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:55.320932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbc0b8-b91d-414d-8e3c-1b5095acf7bd None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.321644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbc0b8-b91d-414d-8e3c-1b5095acf7bd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.321644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbc0b8-b91d-414d-8e3c-1b5095acf7bd None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.321644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 93/185] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:55.321644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbc0b8-b91d-414d-8e3c-1b5095acf7bd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.330408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:55.339491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26c7c4-347c-4545-9219-07216b38a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.340161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26c7c4-347c-4545-9219-07216b38a871 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.340435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26c7c4-347c-4545-9219-07216b38a871 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.340712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26c7c4-347c-4545-9219-07216b38a871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.341582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26c7c4-347c-4545-9219-07216b38a871 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=ab57ebe7c2f9472facbe45d50407af91, user.id=645d7df5139643969242f8464ac102fe) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:55.341900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26c7c4-347c-4545-9219-07216b38a871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:55.343376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26c7c4-347c-4545-9219-07216b38a871 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:55.349158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 93/186] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/role_assignments?user.id=645d7df5139643969242f8464ac102fe&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:55.350311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6def40-b20e-4fcd-a317-39410d6787dc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.350590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6def40-b20e-4fcd-a317-39410d6787dc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.350875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6def40-b20e-4fcd-a317-39410d6787dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.369868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ad6def40-b20e-4fcd-a317-39410d6787dc None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:55.371265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ad6def40-b20e-4fcd-a317-39410d6787dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:55.433222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbc0b8-b91d-414d-8e3c-1b5095acf7bd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.448949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 94/187] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:55.450360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6def40-b20e-4fcd-a317-39410d6787dc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.482948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049cdcf-8874-46e4-acff-70b7c6e07c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.483676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049cdcf-8874-46e4-acff-70b7c6e07c8f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.483980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049cdcf-8874-46e4-acff-70b7c6e07c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.484247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049cdcf-8874-46e4-acff-70b7c6e07c8f None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.485023 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049cdcf-8874-46e4-acff-70b7c6e07c8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:55.485341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049cdcf-8874-46e4-acff-70b7c6e07c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:55.486807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049cdcf-8874-46e4-acff-70b7c6e07c8f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:55.496546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 94/188] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:55.500429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049cdcf-8874-46e4-acff-70b7c6e07c8f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.559370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:55.565237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6def40-b20e-4fcd-a317-39410d6787dc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.571795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:55.601529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f537988-7cbd-47c7-8b79-50bf541ebf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.602412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537988-7cbd-47c7-8b79-50bf541ebf41 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.602698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537988-7cbd-47c7-8b79-50bf541ebf41 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.602982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537988-7cbd-47c7-8b79-50bf541ebf41 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.604182 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f537988-7cbd-47c7-8b79-50bf541ebf41 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:55.605658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f537988-7cbd-47c7-8b79-50bf541ebf41 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:55.605765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049cdcf-8874-46e4-acff-70b7c6e07c8f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.607420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 95/189] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:55.607420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915ba36-512e-4987-af8f-3d2f7afce7e4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.607420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915ba36-512e-4987-af8f-3d2f7afce7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.607420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915ba36-512e-4987-af8f-3d2f7afce7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.624932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1915ba36-512e-4987-af8f-3d2f7afce7e4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:55.626124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1915ba36-512e-4987-af8f-3d2f7afce7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:55.637964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd1a41-dd22-483b-9244-2bc5c6239cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.638683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd1a41-dd22-483b-9244-2bc5c6239cdc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.638967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd1a41-dd22-483b-9244-2bc5c6239cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.639243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd1a41-dd22-483b-9244-2bc5c6239cdc None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.640032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd1a41-dd22-483b-9244-2bc5c6239cdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:55.640338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd1a41-dd22-483b-9244-2bc5c6239cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:55.641741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd1a41-dd22-483b-9244-2bc5c6239cdc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:55.649686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 95/190] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:55.650456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd1a41-dd22-483b-9244-2bc5c6239cdc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.691014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 96/191] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:55.692070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915ba36-512e-4987-af8f-3d2f7afce7e4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.751591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd1a41-dd22-483b-9244-2bc5c6239cdc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.778815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:55.782577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:55.786528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e5035-79ba-44df-801a-5136def8c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.787369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e5035-79ba-44df-801a-5136def8c807 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.787684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e5035-79ba-44df-801a-5136def8c807 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.787959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e5035-79ba-44df-801a-5136def8c807 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.790977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915ba36-512e-4987-af8f-3d2f7afce7e4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.798055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e5035-79ba-44df-801a-5136def8c807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-445e5035-79ba-44df-801a-5136def8c807 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:55.806197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:55.807028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 96/192] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:55.812827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e5035-79ba-44df-801a-5136def8c807 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.824935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fb510-21a8-4415-a04f-b17e2cab8ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.825826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fb510-21a8-4415-a04f-b17e2cab8ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.826152 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fb510-21a8-4415-a04f-b17e2cab8ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.826430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fb510-21a8-4415-a04f-b17e2cab8ea0 None admin] PATH_INFO: `/v3/users/74bb1fcf11c04f73a0b14c3144de90e4` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.827661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fb510-21a8-4415-a04f-b17e2cab8ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=74bb1fcf11c04f73a0b14c3144de90e4)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:55.829113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fb510-21a8-4415-a04f-b17e2cab8ea0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:55.830750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 97/193] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/users/74bb1fcf11c04f73a0b14c3144de90e4 => generated 289 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:55.835271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fb510-21a8-4415-a04f-b17e2cab8ea0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.911208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e5035-79ba-44df-801a-5136def8c807 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.934179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fb510-21a8-4415-a04f-b17e2cab8ea0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:55.943980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94920a3-5427-4c83-8f88-03add731f94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.944640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94920a3-5427-4c83-8f88-03add731f94b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.944927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94920a3-5427-4c83-8f88-03add731f94b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.945210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94920a3-5427-4c83-8f88-03add731f94b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.946071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94920a3-5427-4c83-8f88-03add731f94b None admin] RBAC: Adding query filter params (role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=ab57ebe7c2f9472facbe45d50407af91, user.id=6d22fba8da874faab7222240ee9e9e8c) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:55.946386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94920a3-5427-4c83-8f88-03add731f94b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:55.947904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94920a3-5427-4c83-8f88-03add731f94b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:55.954416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 97/194] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/role_assignments?user.id=6d22fba8da874faab7222240ee9e9e8c&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:55.955213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94920a3-5427-4c83-8f88-03add731f94b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:55.975164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda94a7a-f700-4082-a51a-ce3a5bd7cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:55.976109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94a7a-f700-4082-a51a-ce3a5bd7cb0b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:55.976393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94a7a-f700-4082-a51a-ce3a5bd7cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:55.976681 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94a7a-f700-4082-a51a-ce3a5bd7cb0b None admin] PATH_INFO: `/v3/users/885394bcef08445f91a33940725efb7a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:55.992911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda94a7a-f700-4082-a51a-ce3a5bd7cb0b None admin] RBAC: Authorizing `identity:get_user(user_id=885394bcef08445f91a33940725efb7a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:55.994459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda94a7a-f700-4082-a51a-ce3a5bd7cb0b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:55.996324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 98/195] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/users/885394bcef08445f91a33940725efb7a => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:55.997117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda94a7a-f700-4082-a51a-ce3a5bd7cb0b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.022015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:56.025723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:56.082940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94920a3-5427-4c83-8f88-03add731f94b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.116313 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda94a7a-f700-4082-a51a-ce3a5bd7cb0b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.118700 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea80172-da8d-47a0-97f5-483cbbd021ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.119543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea80172-da8d-47a0-97f5-483cbbd021ea None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.119825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea80172-da8d-47a0-97f5-483cbbd021ea None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.120099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea80172-da8d-47a0-97f5-483cbbd021ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.121265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea80172-da8d-47a0-97f5-483cbbd021ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:56.122769 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea80172-da8d-47a0-97f5-483cbbd021ea None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:56.124430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 98/196] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:56.125295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea80172-da8d-47a0-97f5-483cbbd021ea None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.154616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c28742-5723-41bc-9afc-f3b2d7132d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.155446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c28742-5723-41bc-9afc-f3b2d7132d7c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.155766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c28742-5723-41bc-9afc-f3b2d7132d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.156043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c28742-5723-41bc-9afc-f3b2d7132d7c None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.156807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c28742-5723-41bc-9afc-f3b2d7132d7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:56.157129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c28742-5723-41bc-9afc-f3b2d7132d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:56.158564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c28742-5723-41bc-9afc-f3b2d7132d7c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:56.167249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 99/197] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:27:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:56.168993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c28742-5723-41bc-9afc-f3b2d7132d7c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.233864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea80172-da8d-47a0-97f5-483cbbd021ea None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.251977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:56.256531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:56.269545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c28742-5723-41bc-9afc-f3b2d7132d7c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.269663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5699c0f-b696-406b-bbfa-dcaa552634ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.270274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5699c0f-b696-406b-bbfa-dcaa552634ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.270560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5699c0f-b696-406b-bbfa-dcaa552634ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.270930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5699c0f-b696-406b-bbfa-dcaa552634ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.305120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d7afd-700f-435b-9df1-9159755b3659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.305773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d7afd-700f-435b-9df1-9159755b3659 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.306059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d7afd-700f-435b-9df1-9159755b3659 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.306342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d7afd-700f-435b-9df1-9159755b3659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.338011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5699c0f-b696-406b-bbfa-dcaa552634ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:56.339425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5699c0f-b696-406b-bbfa-dcaa552634ac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:56.373060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d7afd-700f-435b-9df1-9159755b3659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:56.374532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d7afd-700f-435b-9df1-9159755b3659 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:56.435593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 99/198] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:56.436602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5699c0f-b696-406b-bbfa-dcaa552634ac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.457053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:56.461030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:56.477031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 100/199] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:56.477986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d7afd-700f-435b-9df1-9159755b3659 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.536272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5699c0f-b696-406b-bbfa-dcaa552634ac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.570977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dc150-ceee-4011-a03f-f145a49d34c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.571687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dc150-ceee-4011-a03f-f145a49d34c5 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.571971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dc150-ceee-4011-a03f-f145a49d34c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.572248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dc150-ceee-4011-a03f-f145a49d34c5 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.572604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d7afd-700f-435b-9df1-9159755b3659 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.573243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dc150-ceee-4011-a03f-f145a49d34c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:56.575042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dc150-ceee-4011-a03f-f145a49d34c5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:56.602019 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 100/200] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:27:56 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:27:56.602945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dc150-ceee-4011-a03f-f145a49d34c5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.608877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31010ce-b10c-46cf-9078-82b940b4ee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.610393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31010ce-b10c-46cf-9078-82b940b4ee52 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.611100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31010ce-b10c-46cf-9078-82b940b4ee52 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.611100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31010ce-b10c-46cf-9078-82b940b4ee52 None admin] PATH_INFO: `/v3/users/885394bcef08445f91a33940725efb7a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.612639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31010ce-b10c-46cf-9078-82b940b4ee52 None admin] RBAC: Authorizing `identity:get_user(user_id=885394bcef08445f91a33940725efb7a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:56.615271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31010ce-b10c-46cf-9078-82b940b4ee52 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:56.616961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 101/201] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:56 2022] GET /identity/v3/users/885394bcef08445f91a33940725efb7a => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:56.617781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31010ce-b10c-46cf-9078-82b940b4ee52 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.671541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:56.699636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:56.723742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dc150-ceee-4011-a03f-f145a49d34c5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31010ce-b10c-46cf-9078-82b940b4ee52 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.763153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f62ad-27d9-4aba-a2d0-b0938a0e4788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.764590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f62ad-27d9-4aba-a2d0-b0938a0e4788 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.764890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f62ad-27d9-4aba-a2d0-b0938a0e4788 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.765467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f62ad-27d9-4aba-a2d0-b0938a0e4788 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.766790 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f62ad-27d9-4aba-a2d0-b0938a0e4788 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:56.768302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f62ad-27d9-4aba-a2d0-b0938a0e4788 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:56.770182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 101/202] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:56 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:56.770805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f62ad-27d9-4aba-a2d0-b0938a0e4788 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.773413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a3f49-efa1-4112-b7bf-8c71e6a46263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.774316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a3f49-efa1-4112-b7bf-8c71e6a46263 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.774600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a3f49-efa1-4112-b7bf-8c71e6a46263 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.775532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a3f49-efa1-4112-b7bf-8c71e6a46263 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.776427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a3f49-efa1-4112-b7bf-8c71e6a46263 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:56.777889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a3f49-efa1-4112-b7bf-8c71e6a46263 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:56.781110 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 102/203] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:56 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:56.781110 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a3f49-efa1-4112-b7bf-8c71e6a46263 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.899100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f62ad-27d9-4aba-a2d0-b0938a0e4788 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.903108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a3f49-efa1-4112-b7bf-8c71e6a46263 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:56.921705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:56.941719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2b51c-9837-4eaf-9183-b65aeaba3735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.942354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0d0b3-d55e-47dc-97d8-ab1406c57b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:56.942462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2b51c-9837-4eaf-9183-b65aeaba3735 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.942618 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2b51c-9837-4eaf-9183-b65aeaba3735 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.942899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2b51c-9837-4eaf-9183-b65aeaba3735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.943012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d0b3-d55e-47dc-97d8-ab1406c57b3d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:56.943313 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d0b3-d55e-47dc-97d8-ab1406c57b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:56.943643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d0b3-d55e-47dc-97d8-ab1406c57b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:56.944831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0d0b3-d55e-47dc-97d8-ab1406c57b3d None admin] RBAC: Adding query filter params (role.id=b27dca438b9d4942b4739e98f7796831, scope.project.id=ab57ebe7c2f9472facbe45d50407af91, user.id=74bb1fcf11c04f73a0b14c3144de90e4) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:56.944831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0d0b3-d55e-47dc-97d8-ab1406c57b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:56.946256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0d0b3-d55e-47dc-97d8-ab1406c57b3d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:56.954325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 103/204] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:56 2022] GET /identity/v3/role_assignments?user.id=74bb1fcf11c04f73a0b14c3144de90e4&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=b27dca438b9d4942b4739e98f7796831 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:56.955158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0d0b3-d55e-47dc-97d8-ab1406c57b3d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:56.961947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:57.025239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2b51c-9837-4eaf-9183-b65aeaba3735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:57.026788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2b51c-9837-4eaf-9183-b65aeaba3735 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:57.078209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0d0b3-d55e-47dc-97d8-ab1406c57b3d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:57.115930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca5d18-693e-424c-a3bb-6f2be5768f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:57.116958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5d18-693e-424c-a3bb-6f2be5768f32 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:57.117463 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5d18-693e-424c-a3bb-6f2be5768f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:57.117931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5d18-693e-424c-a3bb-6f2be5768f32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:57.119078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca5d18-693e-424c-a3bb-6f2be5768f32 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b, user.id=885394bcef08445f91a33940725efb7a) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:57.127846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca5d18-693e-424c-a3bb-6f2be5768f32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:57.129476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca5d18-693e-424c-a3bb-6f2be5768f32 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:57.141744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 102/205] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:57.146545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 104/206] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:56 2022] GET /identity/v3/role_assignments?user.id=885394bcef08445f91a33940725efb7a&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:57.147660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2b51c-9837-4eaf-9183-b65aeaba3735 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:57.223365 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:57.282142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2b51c-9837-4eaf-9183-b65aeaba3735 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:57.319021 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71bec4-126f-49ec-a63c-21d4a1dc173d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:57.319945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71bec4-126f-49ec-a63c-21d4a1dc173d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:57.320499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71bec4-126f-49ec-a63c-21d4a1dc173d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:57.320778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71bec4-126f-49ec-a63c-21d4a1dc173d None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:57.322056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71bec4-126f-49ec-a63c-21d4a1dc173d None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:57.323630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71bec4-126f-49ec-a63c-21d4a1dc173d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:57.325839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 103/207] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:27:57 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:57.331787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca5d18-693e-424c-a3bb-6f2be5768f32 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:57.397389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:57.462016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca5d18-693e-424c-a3bb-6f2be5768f32 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:57.502268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03efd31-bff4-4c6a-84f0-a7a7b44e92a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:57.503231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03efd31-bff4-4c6a-84f0-a7a7b44e92a8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:57.503724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03efd31-bff4-4c6a-84f0-a7a7b44e92a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:57.504171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03efd31-bff4-4c6a-84f0-a7a7b44e92a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:57.505176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03efd31-bff4-4c6a-84f0-a7a7b44e92a8 None admin] RBAC: Adding query filter params (role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b, user.id=885394bcef08445f91a33940725efb7a) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:57.505684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03efd31-bff4-4c6a-84f0-a7a7b44e92a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:57.507317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03efd31-bff4-4c6a-84f0-a7a7b44e92a8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:57.513104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 105/208] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:27:57 2022] GET /identity/v3/role_assignments?user.id=885394bcef08445f91a33940725efb7a&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:58.789971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4bc6c-d389-4e80-975b-6c009d8bd7f9 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:58.790992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4bc6c-d389-4e80-975b-6c009d8bd7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:58.791532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4bc6c-d389-4e80-975b-6c009d8bd7f9 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:58.793240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 104/209] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:58.798894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080c817-93eb-4f71-b3b5-b7065db7dd5f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:58.799760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080c817-93eb-4f71-b3b5-b7065db7dd5f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:58.800233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080c817-93eb-4f71-b3b5-b7065db7dd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:58.817221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b080c817-93eb-4f71-b3b5-b7065db7dd5f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:58.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b080c817-93eb-4f71-b3b5-b7065db7dd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:58.885528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 106/210] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:58.908905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4bc6c-d389-4e80-975b-6c009d8bd7f9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:58.940762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab531bf-b497-4998-8ef5-225df364c0c2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:58.941087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab531bf-b497-4998-8ef5-225df364c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:58.941339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab531bf-b497-4998-8ef5-225df364c0c2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:58.942736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 107/211] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:27:58.947718 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebbd62-fd63-4356-a5d1-f51fae831e37 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:58.948009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebbd62-fd63-4356-a5d1-f51fae831e37 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:58.948291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebbd62-fd63-4356-a5d1-f51fae831e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:58.965139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-45ebbd62-fd63-4356-a5d1-f51fae831e37 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:58.966367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-45ebbd62-fd63-4356-a5d1-f51fae831e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:59.009390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:59.018307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4bc6c-d389-4e80-975b-6c009d8bd7f9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:59.035344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:59.054260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 108/212] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:27:59.069197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62e7b3-f301-46b8-b90b-db1818103e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:59.070216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62e7b3-f301-46b8-b90b-db1818103e5d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.070669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62e7b3-f301-46b8-b90b-db1818103e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.071109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62e7b3-f301-46b8-b90b-db1818103e5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.078118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebbd62-fd63-4356-a5d1-f51fae831e37 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:59.083550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62e7b3-f301-46b8-b90b-db1818103e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-de62e7b3-f301-46b8-b90b-db1818103e5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:59.092875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:59.093734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:27:59.094532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 105/213] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:59.099995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62e7b3-f301-46b8-b90b-db1818103e5d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:59.209889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebbd62-fd63-4356-a5d1-f51fae831e37 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:59.218918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62e7b3-f301-46b8-b90b-db1818103e5d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:59.255002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26a372-5950-4744-84cd-a106b5ebfe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:59.255449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2507a9-8748-4f3b-a77c-20cfab280a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:59.256298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2507a9-8748-4f3b-a77c-20cfab280a49 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.256495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26a372-5950-4744-84cd-a106b5ebfe60 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.256589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2507a9-8748-4f3b-a77c-20cfab280a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.256850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2507a9-8748-4f3b-a77c-20cfab280a49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.257264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26a372-5950-4744-84cd-a106b5ebfe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.257812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26a372-5950-4744-84cd-a106b5ebfe60 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.258214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2507a9-8748-4f3b-a77c-20cfab280a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:59.258925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26a372-5950-4744-84cd-a106b5ebfe60 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:59.259400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26a372-5950-4744-84cd-a106b5ebfe60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:59.259897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2507a9-8748-4f3b-a77c-20cfab280a49 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:59.261192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26a372-5950-4744-84cd-a106b5ebfe60 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:59.263526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 109/214] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:27:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:59.265305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed49a-7a8f-408e-8da5-866a735aa814 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.265591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed49a-7a8f-408e-8da5-866a735aa814 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.265966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed49a-7a8f-408e-8da5-866a735aa814 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.267508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 110/215] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:27:59.271868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed49a-7a8f-408e-8da5-866a735aa814 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:59.274428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 106/216] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:27:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:27:59.279958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cae86-d93e-41ce-aa8d-8f73a1d604c9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.280275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cae86-d93e-41ce-aa8d-8f73a1d604c9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.280568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cae86-d93e-41ce-aa8d-8f73a1d604c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.296515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-438cae86-d93e-41ce-aa8d-8f73a1d604c9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:59.297701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-438cae86-d93e-41ce-aa8d-8f73a1d604c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:59.314811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:59.328938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:59.368745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 107/217] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:59.369575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cae86-d93e-41ce-aa8d-8f73a1d604c9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:59.387370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed49a-7a8f-408e-8da5-866a735aa814 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:59.422595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068921ad-f8e8-4866-bfeb-da7207a7c6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:59.423272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068921ad-f8e8-4866-bfeb-da7207a7c6e8 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.423582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068921ad-f8e8-4866-bfeb-da7207a7c6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.423852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068921ad-f8e8-4866-bfeb-da7207a7c6e8 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.424570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068921ad-f8e8-4866-bfeb-da7207a7c6e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:59.426014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068921ad-f8e8-4866-bfeb-da7207a7c6e8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:59.448782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 111/218] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:27:59 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:27:59.449906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068921ad-f8e8-4866-bfeb-da7207a7c6e8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:59.492276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cae86-d93e-41ce-aa8d-8f73a1d604c9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:59.542056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3869259-2bd3-419c-a695-4cd097fd208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:59.543224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3869259-2bd3-419c-a695-4cd097fd208e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.543717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3869259-2bd3-419c-a695-4cd097fd208e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.544132 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3869259-2bd3-419c-a695-4cd097fd208e None admin] PATH_INFO: `/v3/users/482df145b8c14524aa96fdfbd4ceb794` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.565203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3869259-2bd3-419c-a695-4cd097fd208e None admin] RBAC: Authorizing `identity:get_user(user_id=482df145b8c14524aa96fdfbd4ceb794)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:59.566908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:59.567748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3869259-2bd3-419c-a695-4cd097fd208e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:59.570231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 108/219] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:59 2022] GET /identity/v3/users/482df145b8c14524aa96fdfbd4ceb794 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:59.570231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121e46e-a018-4e6a-b6b8-4b919a24b74c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.570231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121e46e-a018-4e6a-b6b8-4b919a24b74c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.570231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121e46e-a018-4e6a-b6b8-4b919a24b74c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.575447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 109/220] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:27:59.576542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121e46e-a018-4e6a-b6b8-4b919a24b74c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:59.583435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068921ad-f8e8-4866-bfeb-da7207a7c6e8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:59.604770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:59.621653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514737e-8bb5-44f8-956a-b81ae58b06f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:59.622798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514737e-8bb5-44f8-956a-b81ae58b06f5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.622798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514737e-8bb5-44f8-956a-b81ae58b06f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.623045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514737e-8bb5-44f8-956a-b81ae58b06f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.635670 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514737e-8bb5-44f8-956a-b81ae58b06f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1514737e-8bb5-44f8-956a-b81ae58b06f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:27:59.642906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:27:59.643821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:27:59.644615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 112/221] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:27:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:27:59.650150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528a5e2-3997-47d9-8123-f9df6f741532 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.650448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528a5e2-3997-47d9-8123-f9df6f741532 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.650728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528a5e2-3997-47d9-8123-f9df6f741532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.672880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3528a5e2-3997-47d9-8123-f9df6f741532 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:27:59.674077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3528a5e2-3997-47d9-8123-f9df6f741532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:27:59.697358 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121e46e-a018-4e6a-b6b8-4b919a24b74c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:59.745062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545580f-41a2-41d0-9511-f8cd9623b83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:59.745719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545580f-41a2-41d0-9511-f8cd9623b83d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.746003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545580f-41a2-41d0-9511-f8cd9623b83d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.746282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545580f-41a2-41d0-9511-f8cd9623b83d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.756229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 113/222] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:27:59.756923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528a5e2-3997-47d9-8123-f9df6f741532 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:59.841876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545580f-41a2-41d0-9511-f8cd9623b83d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:59.844112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545580f-41a2-41d0-9511-f8cd9623b83d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:59.861344 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:59.877570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:27:59.879095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528a5e2-3997-47d9-8123-f9df6f741532 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:27:59.921106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b64740-e1be-44f0-9e11-7b871a3a6721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:27:59.921794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b64740-e1be-44f0-9e11-7b871a3a6721 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:27:59.922084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b64740-e1be-44f0-9e11-7b871a3a6721 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:27:59.922388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b64740-e1be-44f0-9e11-7b871a3a6721 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:27:59.923149 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b64740-e1be-44f0-9e11-7b871a3a6721 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:27:59.923482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b64740-e1be-44f0-9e11-7b871a3a6721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:27:59.924903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b64740-e1be-44f0-9e11-7b871a3a6721 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:27:59.935878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 114/223] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:27:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:27:59.936833 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b64740-e1be-44f0-9e11-7b871a3a6721 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:27:59.991011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 110/224] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:27:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:27:59.992311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545580f-41a2-41d0-9511-f8cd9623b83d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:00.062084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b64740-e1be-44f0-9e11-7b871a3a6721 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:00.101647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0ef34-92cf-4687-8496-135412b18258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:00.102489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0ef34-92cf-4687-8496-135412b18258 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:00.102672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0ef34-92cf-4687-8496-135412b18258 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:00.102951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0ef34-92cf-4687-8496-135412b18258 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:00.114652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0ef34-92cf-4687-8496-135412b18258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:00.116068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545580f-41a2-41d0-9511-f8cd9623b83d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-92b0ef34-92cf-4687-8496-135412b18258 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:00.123275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:00.124198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:00.124967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 115/225] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:27:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:00.125589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0ef34-92cf-4687-8496-135412b18258 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:00.140407 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:00.167320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2d083-bb55-4507-8816-11328e9dc9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:00.169644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2d083-bb55-4507-8816-11328e9dc9ec None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:00.170104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2d083-bb55-4507-8816-11328e9dc9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:00.170521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2d083-bb55-4507-8816-11328e9dc9ec None admin] PATH_INFO: `/v3/users/885394bcef08445f91a33940725efb7a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:00.175633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:00.176699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2d083-bb55-4507-8816-11328e9dc9ec None admin] RBAC: Authorizing `identity:get_user(user_id=885394bcef08445f91a33940725efb7a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:00.178765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2d083-bb55-4507-8816-11328e9dc9ec None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:00.182165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 111/226] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:27:59 2022] GET /identity/v3/users/885394bcef08445f91a33940725efb7a => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:00.183854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2d083-bb55-4507-8816-11328e9dc9ec None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:00.248339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0ef34-92cf-4687-8496-135412b18258 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:00.285821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dda2e5-8700-4a5c-8f9a-a0355603bdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:00.286722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dda2e5-8700-4a5c-8f9a-a0355603bdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:00.287053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dda2e5-8700-4a5c-8f9a-a0355603bdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:00.287428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dda2e5-8700-4a5c-8f9a-a0355603bdf5 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:00.288754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dda2e5-8700-4a5c-8f9a-a0355603bdf5 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:00.290410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dda2e5-8700-4a5c-8f9a-a0355603bdf5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:00.292786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 116/227] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:00 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:00.293574 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dda2e5-8700-4a5c-8f9a-a0355603bdf5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:00.295969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2d083-bb55-4507-8816-11328e9dc9ec None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:00.330649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3874063-c989-41ad-ac8a-6fc4b69ff5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:00.331589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3874063-c989-41ad-ac8a-6fc4b69ff5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:00.332008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3874063-c989-41ad-ac8a-6fc4b69ff5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:00.332446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3874063-c989-41ad-ac8a-6fc4b69ff5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:00.333329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3874063-c989-41ad-ac8a-6fc4b69ff5b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:00.333775 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3874063-c989-41ad-ac8a-6fc4b69ff5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:00.335309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3874063-c989-41ad-ac8a-6fc4b69ff5b5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:00.344204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 112/228] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:00.349963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3874063-c989-41ad-ac8a-6fc4b69ff5b5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:00.380705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:00.408369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dda2e5-8700-4a5c-8f9a-a0355603bdf5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:00.409006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:00.450868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1a779-1463-4ac7-9f0e-0eb3dc7a1c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:00.451968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a779-1463-4ac7-9f0e-0eb3dc7a1c32 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:00.452259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a779-1463-4ac7-9f0e-0eb3dc7a1c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:00.452539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a779-1463-4ac7-9f0e-0eb3dc7a1c32 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:00.453729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1a779-1463-4ac7-9f0e-0eb3dc7a1c32 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:00.454654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3874063-c989-41ad-ac8a-6fc4b69ff5b5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:00.455193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1a779-1463-4ac7-9f0e-0eb3dc7a1c32 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:00.457478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 117/229] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:00 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:00.458315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1a779-1463-4ac7-9f0e-0eb3dc7a1c32 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:00.487835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b062e9b-f91f-4778-ad5a-722932cfe96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:00.488494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b062e9b-f91f-4778-ad5a-722932cfe96f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:00.488780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b062e9b-f91f-4778-ad5a-722932cfe96f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:00.489055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b062e9b-f91f-4778-ad5a-722932cfe96f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:00.489886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b062e9b-f91f-4778-ad5a-722932cfe96f None admin] RBAC: Adding query filter params (role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b, user.id=482df145b8c14524aa96fdfbd4ceb794) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:00.490197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b062e9b-f91f-4778-ad5a-722932cfe96f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:00.491684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b062e9b-f91f-4778-ad5a-722932cfe96f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:00.497245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 113/230] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:00 2022] GET /identity/v3/role_assignments?user.id=482df145b8c14524aa96fdfbd4ceb794&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=0428a514a20043c680400ebf6498c74e => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:00.497953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b062e9b-f91f-4778-ad5a-722932cfe96f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:00.574499 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1a779-1463-4ac7-9f0e-0eb3dc7a1c32 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:00.600416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b062e9b-f91f-4778-ad5a-722932cfe96f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:00.614743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff9f39-ed43-4345-9894-a4e2e58bee36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:00.615809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff9f39-ed43-4345-9894-a4e2e58bee36 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:00.616103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff9f39-ed43-4345-9894-a4e2e58bee36 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:00.616389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff9f39-ed43-4345-9894-a4e2e58bee36 None admin] PATH_INFO: `/v3/users/885394bcef08445f91a33940725efb7a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:00.617977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff9f39-ed43-4345-9894-a4e2e58bee36 None admin] RBAC: Authorizing `identity:get_user(user_id=885394bcef08445f91a33940725efb7a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:00.619006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:00.620545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff9f39-ed43-4345-9894-a4e2e58bee36 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:00.622854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 118/231] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:00 2022] GET /identity/v3/users/885394bcef08445f91a33940725efb7a => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:00.638510 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff9f39-ed43-4345-9894-a4e2e58bee36 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:00.643529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e7982-4651-4c0f-9546-328f715ca205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:00.644459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e7982-4651-4c0f-9546-328f715ca205 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:00.644741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e7982-4651-4c0f-9546-328f715ca205 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:00.645014 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e7982-4651-4c0f-9546-328f715ca205 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/users/885394bcef08445f91a33940725efb7a/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:00.647221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e7982-4651-4c0f-9546-328f715ca205 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1cafe4585e6448983aef6e3d236bd3b, user_id=885394bcef08445f91a33940725efb7a, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:00.648773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e7982-4651-4c0f-9546-328f715ca205 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:00.664412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:00.667874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 114/232] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:28:00 2022] PUT /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/users/885394bcef08445f91a33940725efb7a/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:00.785126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff9f39-ed43-4345-9894-a4e2e58bee36 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:00.836084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4588a3-eb9d-4f44-9ff6-8878c7a99b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:00.837009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4588a3-eb9d-4f44-9ff6-8878c7a99b6a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:00.837296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4588a3-eb9d-4f44-9ff6-8878c7a99b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:00.837580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4588a3-eb9d-4f44-9ff6-8878c7a99b6a None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:00.844469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4588a3-eb9d-4f44-9ff6-8878c7a99b6a None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:00.845988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4588a3-eb9d-4f44-9ff6-8878c7a99b6a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:00.848079 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 119/233] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:00 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:00.853804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e7982-4651-4c0f-9546-328f715ca205 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:00.977013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e7982-4651-4c0f-9546-328f715ca205 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:01.015374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96731c1-1ca9-4446-9406-59584b29941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:01.016350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96731c1-1ca9-4446-9406-59584b29941b None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:01.016628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96731c1-1ca9-4446-9406-59584b29941b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:01.016904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96731c1-1ca9-4446-9406-59584b29941b None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/users/885394bcef08445f91a33940725efb7a/roles/32bea78dbfe74b56bd5c1f113586c690` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:01.019202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96731c1-1ca9-4446-9406-59584b29941b None admin] RBAC: Authorizing `identity:create_grant(project_id=d1cafe4585e6448983aef6e3d236bd3b, user_id=885394bcef08445f91a33940725efb7a, role_id=32bea78dbfe74b56bd5c1f113586c690)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:01.020718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96731c1-1ca9-4446-9406-59584b29941b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:01.040937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 115/234] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:28:00 2022] PUT /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/users/885394bcef08445f91a33940725efb7a/roles/32bea78dbfe74b56bd5c1f113586c690 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:01.893642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26416d9-1419-44e2-94e3-9b2cafd6ced7 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:01.893642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26416d9-1419-44e2-94e3-9b2cafd6ced7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:01.893642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26416d9-1419-44e2-94e3-9b2cafd6ced7 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:01.894904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 120/235] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:01.900403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b814a-2968-40bd-a8a9-82acf256bec4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:01.900869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b814a-2968-40bd-a8a9-82acf256bec4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:01.901019 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b814a-2968-40bd-a8a9-82acf256bec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:01.918384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-218b814a-2968-40bd-a8a9-82acf256bec4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:01.919736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-218b814a-2968-40bd-a8a9-82acf256bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:01.959240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:01.998701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e1fee-d15e-436d-9285-1287327e4593 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:01.999031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e1fee-d15e-436d-9285-1287327e4593 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:01.999294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e1fee-d15e-436d-9285-1287327e4593 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:02.000822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 121/236] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:02.007539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee0ab8-b511-43b8-b226-f2c679eac487 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:02.008064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee0ab8-b511-43b8-b226-f2c679eac487 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:02.008544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee0ab8-b511-43b8-b226-f2c679eac487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:02.008646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 116/237] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:02.025794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e3ee0ab8-b511-43b8-b226-f2c679eac487 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:02.027331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e3ee0ab8-b511-43b8-b226-f2c679eac487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:02.050594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b814a-2968-40bd-a8a9-82acf256bec4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:02.109186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 122/238] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:02.133019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee0ab8-b511-43b8-b226-f2c679eac487 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:02.144166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:02.233814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b814a-2968-40bd-a8a9-82acf256bec4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:02.238917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee0ab8-b511-43b8-b226-f2c679eac487 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:02.275256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783a455-1177-40e5-a5e7-acc36a09b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:02.276478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783a455-1177-40e5-a5e7-acc36a09b4df None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:02.276962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783a455-1177-40e5-a5e7-acc36a09b4df None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:02.277434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783a455-1177-40e5-a5e7-acc36a09b4df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:02.289743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783a455-1177-40e5-a5e7-acc36a09b4df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8783a455-1177-40e5-a5e7-acc36a09b4df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:02.298127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:02.299005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:02.299824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:02.302415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 123/239] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:02.308749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783a455-1177-40e5-a5e7-acc36a09b4df None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:02.308874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a1f47-f126-4efc-b3bd-4cf416568c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:02.309589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1f47-f126-4efc-b3bd-4cf416568c62 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:02.309881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1f47-f126-4efc-b3bd-4cf416568c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:02.310247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1f47-f126-4efc-b3bd-4cf416568c62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:02.312235 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a1f47-f126-4efc-b3bd-4cf416568c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:02.313722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a1f47-f126-4efc-b3bd-4cf416568c62 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:02.316463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 117/240] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:02.320888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a1f47-f126-4efc-b3bd-4cf416568c62 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:02.377148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:02.411954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:02.420445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783a455-1177-40e5-a5e7-acc36a09b4df None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:02.483433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a1f47-f126-4efc-b3bd-4cf416568c62 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:02.512451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba56f1-4f95-4808-ba42-a366408df7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:02.513422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba56f1-4f95-4808-ba42-a366408df7db None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:02.513422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba56f1-4f95-4808-ba42-a366408df7db None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:02.513704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba56f1-4f95-4808-ba42-a366408df7db None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:02.514466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba56f1-4f95-4808-ba42-a366408df7db None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:02.514761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba56f1-4f95-4808-ba42-a366408df7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:02.520292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba56f1-4f95-4808-ba42-a366408df7db None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:02.531543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 124/241] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:02.536880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80215f-f35f-4872-b87b-5a041af5a4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:02.537592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80215f-f35f-4872-b87b-5a041af5a4d2 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:02.537883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80215f-f35f-4872-b87b-5a041af5a4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:02.538164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80215f-f35f-4872-b87b-5a041af5a4d2 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:02.538632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba56f1-4f95-4808-ba42-a366408df7db None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:02.539053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80215f-f35f-4872-b87b-5a041af5a4d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:02.540554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80215f-f35f-4872-b87b-5a041af5a4d2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:02.590334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 118/242] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:28:02 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:28:02.650465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba56f1-4f95-4808-ba42-a366408df7db None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:02.680913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:02.688113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5ad8c-bd35-4264-ba55-0de9d951082a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:02.689352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5ad8c-bd35-4264-ba55-0de9d951082a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:02.689856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5ad8c-bd35-4264-ba55-0de9d951082a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:02.690329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5ad8c-bd35-4264-ba55-0de9d951082a None admin] PATH_INFO: `/v3/users/63b6fed1fb584dd7ba5578662c298ffd` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:02.706919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5ad8c-bd35-4264-ba55-0de9d951082a None admin] RBAC: Authorizing `identity:get_user(user_id=63b6fed1fb584dd7ba5578662c298ffd)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:02.708813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5ad8c-bd35-4264-ba55-0de9d951082a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:02.711087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 125/243] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:02 2022] GET /identity/v3/users/63b6fed1fb584dd7ba5578662c298ffd => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:02.716790 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80215f-f35f-4872-b87b-5a041af5a4d2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:02.833306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:02.845762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80215f-f35f-4872-b87b-5a041af5a4d2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:02.882489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33865217-75c1-4efd-babc-36b5b3f8661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:02.883582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33865217-75c1-4efd-babc-36b5b3f8661a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:02.884072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33865217-75c1-4efd-babc-36b5b3f8661a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:02.884545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33865217-75c1-4efd-babc-36b5b3f8661a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:02.955429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33865217-75c1-4efd-babc-36b5b3f8661a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:02.957744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33865217-75c1-4efd-babc-36b5b3f8661a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:03.056357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:03.073303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 119/244] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:03.079447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5ad8c-bd35-4264-ba55-0de9d951082a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:03.093787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad771fe-6cd9-4bc2-99a2-ba0e88bf79d2 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.094080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad771fe-6cd9-4bc2-99a2-ba0e88bf79d2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.094350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad771fe-6cd9-4bc2-99a2-ba0e88bf79d2 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.097385 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 120/245] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:03.101666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975296c-dd81-4333-b784-9f5fecf78f3d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.101954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975296c-dd81-4333-b784-9f5fecf78f3d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.102229 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975296c-dd81-4333-b784-9f5fecf78f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.118962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6975296c-dd81-4333-b784-9f5fecf78f3d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:03.120603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6975296c-dd81-4333-b784-9f5fecf78f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:03.205345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 121/246] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:03.212846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5ad8c-bd35-4264-ba55-0de9d951082a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:03.230458 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975296c-dd81-4333-b784-9f5fecf78f3d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:03.267746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478cafe-0c3b-49a6-829e-ebd91041abd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:03.269229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478cafe-0c3b-49a6-829e-ebd91041abd8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.269733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478cafe-0c3b-49a6-829e-ebd91041abd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.271125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478cafe-0c3b-49a6-829e-ebd91041abd8 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.272575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478cafe-0c3b-49a6-829e-ebd91041abd8 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:03.274232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478cafe-0c3b-49a6-829e-ebd91041abd8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:03.277138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 126/247] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:03 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:03.284624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478cafe-0c3b-49a6-829e-ebd91041abd8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:03.364725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:03.364915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975296c-dd81-4333-b784-9f5fecf78f3d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:03.390407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:03.401406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129d7a7-d221-4c8b-916e-c43efdb9d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:03.402330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129d7a7-d221-4c8b-916e-c43efdb9d635 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.402675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129d7a7-d221-4c8b-916e-c43efdb9d635 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.403007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129d7a7-d221-4c8b-916e-c43efdb9d635 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.414828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129d7a7-d221-4c8b-916e-c43efdb9d635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f129d7a7-d221-4c8b-916e-c43efdb9d635 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:03.423869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:03.425570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:03.427092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 122/248] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:03.430041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443a352-b6e8-4475-9a92-0239d914d982 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.430041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443a352-b6e8-4475-9a92-0239d914d982 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.430041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443a352-b6e8-4475-9a92-0239d914d982 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.430041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 123/249] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:03.431294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478cafe-0c3b-49a6-829e-ebd91041abd8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:03.435116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2297d17-db37-4648-a214-f68dac57bf41 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.435116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2297d17-db37-4648-a214-f68dac57bf41 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.435658 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2297d17-db37-4648-a214-f68dac57bf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.457056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c2297d17-db37-4648-a214-f68dac57bf41 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:03.458286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c2297d17-db37-4648-a214-f68dac57bf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:03.504187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eddaf3-9365-44c3-9a1e-7033d7c5cdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:03.506357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eddaf3-9365-44c3-9a1e-7033d7c5cdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.508069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eddaf3-9365-44c3-9a1e-7033d7c5cdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.508897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eddaf3-9365-44c3-9a1e-7033d7c5cdc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.512049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eddaf3-9365-44c3-9a1e-7033d7c5cdc8 None admin] RBAC: Adding query filter params (role.id=b27dca438b9d4942b4739e98f7796831, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b, user.id=63b6fed1fb584dd7ba5578662c298ffd) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:03.513209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eddaf3-9365-44c3-9a1e-7033d7c5cdc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:03.516738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eddaf3-9365-44c3-9a1e-7033d7c5cdc8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:03.532515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 127/250] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:03 2022] GET /identity/v3/role_assignments?user.id=63b6fed1fb584dd7ba5578662c298ffd&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=b27dca438b9d4942b4739e98f7796831 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:03.534321 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eddaf3-9365-44c3-9a1e-7033d7c5cdc8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:03.536066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 124/251] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:03.579977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2297d17-db37-4648-a214-f68dac57bf41 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:03.775540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2297d17-db37-4648-a214-f68dac57bf41 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:03.789384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:03.811022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ff989-4100-4024-89be-d5b19df5c481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:03.811952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ff989-4100-4024-89be-d5b19df5c481 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.812302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ff989-4100-4024-89be-d5b19df5c481 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.812645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ff989-4100-4024-89be-d5b19df5c481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.823416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ff989-4100-4024-89be-d5b19df5c481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5a3ff989-4100-4024-89be-d5b19df5c481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:03.831981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:03.833643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 125/252] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:03.835195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e38a5-576b-4582-98b6-493f6c0df6aa None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.837432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e38a5-576b-4582-98b6-493f6c0df6aa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.837432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e38a5-576b-4582-98b6-493f6c0df6aa None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.839713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 126/253] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:03.839713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e38a5-576b-4582-98b6-493f6c0df6aa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:03.850775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eddaf3-9365-44c3-9a1e-7033d7c5cdc8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:03.891648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:03.933584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb27f2-2ba3-493c-a0bb-1ddf8fc30b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:03.934277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb27f2-2ba3-493c-a0bb-1ddf8fc30b55 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.934618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb27f2-2ba3-493c-a0bb-1ddf8fc30b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.934888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb27f2-2ba3-493c-a0bb-1ddf8fc30b55 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.935791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb27f2-2ba3-493c-a0bb-1ddf8fc30b55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:03.936100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb27f2-2ba3-493c-a0bb-1ddf8fc30b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:03.937491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb27f2-2ba3-493c-a0bb-1ddf8fc30b55 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:03.956714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 128/254] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:03.963744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae2701-fc87-44c5-8f4e-5a40e3102c1a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:03.964040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae2701-fc87-44c5-8f4e-5a40e3102c1a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:03.964328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae2701-fc87-44c5-8f4e-5a40e3102c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:03.988714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e38a5-576b-4582-98b6-493f6c0df6aa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:03.995372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-beae2701-fc87-44c5-8f4e-5a40e3102c1a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:03.996958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-beae2701-fc87-44c5-8f4e-5a40e3102c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:04.090767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 129/255] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:04.092651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae2701-fc87-44c5-8f4e-5a40e3102c1a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:04.120748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c63b7-5495-4992-a744-1636fbd4f449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:04.121487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c63b7-5495-4992-a744-1636fbd4f449 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:04.121706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c63b7-5495-4992-a744-1636fbd4f449 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:04.121976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c63b7-5495-4992-a744-1636fbd4f449 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:04.122852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c63b7-5495-4992-a744-1636fbd4f449 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:04.123162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c63b7-5495-4992-a744-1636fbd4f449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:04.124602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c63b7-5495-4992-a744-1636fbd4f449 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:04.143154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 127/256] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:04.144106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c63b7-5495-4992-a744-1636fbd4f449 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:04.209868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae2701-fc87-44c5-8f4e-5a40e3102c1a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:04.225769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:04.234387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:04.265934 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c4ed4-e5e8-45a9-ae61-1299a6973655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:04.268106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c4ed4-e5e8-45a9-ae61-1299a6973655 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:04.269891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c4ed4-e5e8-45a9-ae61-1299a6973655 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:04.270376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c4ed4-e5e8-45a9-ae61-1299a6973655 None admin] PATH_INFO: `/v3/users/482df145b8c14524aa96fdfbd4ceb794` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:04.272824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c4ed4-e5e8-45a9-ae61-1299a6973655 None admin] RBAC: Authorizing `identity:get_user(user_id=482df145b8c14524aa96fdfbd4ceb794)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:04.275797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c4ed4-e5e8-45a9-ae61-1299a6973655 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:04.280865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 130/257] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:04 2022] GET /identity/v3/users/482df145b8c14524aa96fdfbd4ceb794 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:04.282766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c4ed4-e5e8-45a9-ae61-1299a6973655 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:04.385878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c63b7-5495-4992-a744-1636fbd4f449 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:04.449538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c4ed4-e5e8-45a9-ae61-1299a6973655 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:04.464853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd7361-cdab-47e4-845e-27a37693acf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:04.466037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7361-cdab-47e4-845e-27a37693acf0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:04.466581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7361-cdab-47e4-845e-27a37693acf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:04.467054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7361-cdab-47e4-845e-27a37693acf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:04.488543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd7361-cdab-47e4-845e-27a37693acf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-eacd7361-cdab-47e4-845e-27a37693acf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:04.508819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:04.509682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:04.510453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 128/258] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:28:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:04.511114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd7361-cdab-47e4-845e-27a37693acf0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:04.517134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6aa4b-8851-4b09-9bbe-95852caf806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:04.518397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6aa4b-8851-4b09-9bbe-95852caf806d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:04.518889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6aa4b-8851-4b09-9bbe-95852caf806d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:04.519360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6aa4b-8851-4b09-9bbe-95852caf806d None admin] PATH_INFO: `/v3/users/885394bcef08445f91a33940725efb7a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:04.520835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6aa4b-8851-4b09-9bbe-95852caf806d None admin] RBAC: Authorizing `identity:get_user(user_id=885394bcef08445f91a33940725efb7a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:04.522632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6aa4b-8851-4b09-9bbe-95852caf806d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:04.524533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 131/259] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:04 2022] GET /identity/v3/users/885394bcef08445f91a33940725efb7a => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:04.525471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6aa4b-8851-4b09-9bbe-95852caf806d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:04.584612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:04.675910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:04.676496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6aa4b-8851-4b09-9bbe-95852caf806d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:04.681532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd7361-cdab-47e4-845e-27a37693acf0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:04.734915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad252c3-216d-4ca7-a95a-6ae8e47ee926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:04.736260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad252c3-216d-4ca7-a95a-6ae8e47ee926 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:04.736547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad252c3-216d-4ca7-a95a-6ae8e47ee926 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:04.736822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad252c3-216d-4ca7-a95a-6ae8e47ee926 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:04.737594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad252c3-216d-4ca7-a95a-6ae8e47ee926 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:04.737899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad252c3-216d-4ca7-a95a-6ae8e47ee926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:04.739337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad252c3-216d-4ca7-a95a-6ae8e47ee926 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:04.756395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 132/260] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:04.757207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41bf74-f2c2-404d-beb8-b21e47a95368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:04.757603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad252c3-216d-4ca7-a95a-6ae8e47ee926 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:04.758221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41bf74-f2c2-404d-beb8-b21e47a95368 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:04.758519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41bf74-f2c2-404d-beb8-b21e47a95368 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:04.758848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41bf74-f2c2-404d-beb8-b21e47a95368 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:04.760522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41bf74-f2c2-404d-beb8-b21e47a95368 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:04.765342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41bf74-f2c2-404d-beb8-b21e47a95368 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:04.767161 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 129/261] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:04 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:04.767965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41bf74-f2c2-404d-beb8-b21e47a95368 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:04.868322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad252c3-216d-4ca7-a95a-6ae8e47ee926 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:04.879894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:04.908697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e1b4c-fe42-445d-ba10-868586e02bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:04.909367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e1b4c-fe42-445d-ba10-868586e02bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:04.909657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e1b4c-fe42-445d-ba10-868586e02bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:04.909976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e1b4c-fe42-445d-ba10-868586e02bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:04.981926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e1b4c-fe42-445d-ba10-868586e02bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:04.983420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e1b4c-fe42-445d-ba10-868586e02bc3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:05.082600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41bf74-f2c2-404d-beb8-b21e47a95368 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:05.091577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 133/262] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:05.092014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e1b4c-fe42-445d-ba10-868586e02bc3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:05.111826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:05.157511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a2596-077e-4713-8a49-6ecd56fa039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:05.159319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a2596-077e-4713-8a49-6ecd56fa039b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:05.159948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a2596-077e-4713-8a49-6ecd56fa039b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:05.160465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a2596-077e-4713-8a49-6ecd56fa039b None admin] PATH_INFO: `/v3/users/885394bcef08445f91a33940725efb7a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:05.165569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a2596-077e-4713-8a49-6ecd56fa039b None admin] RBAC: Authorizing `identity:get_user(user_id=885394bcef08445f91a33940725efb7a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:05.167438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a2596-077e-4713-8a49-6ecd56fa039b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:05.169608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 130/263] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:04 2022] GET /identity/v3/users/885394bcef08445f91a33940725efb7a => generated 283 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:05.170605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a2596-077e-4713-8a49-6ecd56fa039b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:05.221326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e1b4c-fe42-445d-ba10-868586e02bc3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:05.237975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:05.284898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b139cf3-8b1f-400e-ad1a-9d33363367f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:05.285802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b139cf3-8b1f-400e-ad1a-9d33363367f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:05.286083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b139cf3-8b1f-400e-ad1a-9d33363367f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:05.286362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b139cf3-8b1f-400e-ad1a-9d33363367f9 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/users/482df145b8c14524aa96fdfbd4ceb794/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:05.290063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b139cf3-8b1f-400e-ad1a-9d33363367f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1cafe4585e6448983aef6e3d236bd3b, user_id=482df145b8c14524aa96fdfbd4ceb794, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:05.292881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b139cf3-8b1f-400e-ad1a-9d33363367f9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:05.317987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 134/264] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:28:05 2022] PUT /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/users/482df145b8c14524aa96fdfbd4ceb794/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:05.324792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b139cf3-8b1f-400e-ad1a-9d33363367f9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:05.393759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a2596-077e-4713-8a49-6ecd56fa039b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:05.465607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468b0f3-b4ce-48e5-9c82-5adce33edb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:05.467449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468b0f3-b4ce-48e5-9c82-5adce33edb39 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:05.467830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:05.471715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468b0f3-b4ce-48e5-9c82-5adce33edb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:05.472165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468b0f3-b4ce-48e5-9c82-5adce33edb39 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:05.473798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468b0f3-b4ce-48e5-9c82-5adce33edb39 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:05.475437 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468b0f3-b4ce-48e5-9c82-5adce33edb39 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:05.478043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 131/265] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:05 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:05.484276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468b0f3-b4ce-48e5-9c82-5adce33edb39 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:05.540259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b139cf3-8b1f-400e-ad1a-9d33363367f9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:05.614855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd27298-a03c-4900-bbe5-a5d3d2e1ce09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:05.616174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27298-a03c-4900-bbe5-a5d3d2e1ce09 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:05.616458 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27298-a03c-4900-bbe5-a5d3d2e1ce09 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:05.616736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27298-a03c-4900-bbe5-a5d3d2e1ce09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:05.647503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:05.651367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468b0f3-b4ce-48e5-9c82-5adce33edb39 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:05.690585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2a3ad-682c-40ad-a2f9-e26bd78e2155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:05.691249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2a3ad-682c-40ad-a2f9-e26bd78e2155 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:05.691557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2a3ad-682c-40ad-a2f9-e26bd78e2155 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:05.691834 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2a3ad-682c-40ad-a2f9-e26bd78e2155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:05.692683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2a3ad-682c-40ad-a2f9-e26bd78e2155 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b, user.id=885394bcef08445f91a33940725efb7a) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:05.692993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2a3ad-682c-40ad-a2f9-e26bd78e2155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:05.694659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2a3ad-682c-40ad-a2f9-e26bd78e2155 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:05.703586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 132/266] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:05 2022] GET /identity/v3/role_assignments?user.id=885394bcef08445f91a33940725efb7a&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:05.727295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd27298-a03c-4900-bbe5-a5d3d2e1ce09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:05.728925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd27298-a03c-4900-bbe5-a5d3d2e1ce09 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:05.808272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:05.844514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 135/267] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:05.852819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2a3ad-682c-40ad-a2f9-e26bd78e2155 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:05.859046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0f1cf-bdb1-48cc-af72-45ce19055ad5 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:05.859342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0f1cf-bdb1-48cc-af72-45ce19055ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:05.859644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0f1cf-bdb1-48cc-af72-45ce19055ad5 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:05.861170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 136/268] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:05.868004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0b9c9-2e0f-4e12-a5d2-da48af5e5823 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:05.868300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0b9c9-2e0f-4e12-a5d2-da48af5e5823 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:05.868582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0b9c9-2e0f-4e12-a5d2-da48af5e5823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:05.892836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-78a0b9c9-2e0f-4e12-a5d2-da48af5e5823 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:05.894060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78a0b9c9-2e0f-4e12-a5d2-da48af5e5823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:05.985210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 137/269] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:05.986558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba988cd-3c25-4814-b7db-0d9ddf6a5751 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:05.986854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba988cd-3c25-4814-b7db-0d9ddf6a5751 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:05.987204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba988cd-3c25-4814-b7db-0d9ddf6a5751 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:05.988780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 138/270] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:05.994125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e170d-a1ff-48b2-b0d2-af5def1f53f2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:05.994421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e170d-a1ff-48b2-b0d2-af5def1f53f2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:05.994706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e170d-a1ff-48b2-b0d2-af5def1f53f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.012024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2a3ad-682c-40ad-a2f9-e26bd78e2155 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.012758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ad5e170d-a1ff-48b2-b0d2-af5def1f53f2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:06.014098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ad5e170d-a1ff-48b2-b0d2-af5def1f53f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:06.058585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e3a5a-f193-487f-abd6-fde5fae1adad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.059394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e3a5a-f193-487f-abd6-fde5fae1adad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.059777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e3a5a-f193-487f-abd6-fde5fae1adad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.060097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e3a5a-f193-487f-abd6-fde5fae1adad None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.063045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e3a5a-f193-487f-abd6-fde5fae1adad None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:06.063045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e3a5a-f193-487f-abd6-fde5fae1adad None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:06.064679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 133/271] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:05 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:06.065928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b263c76-dd60-4a8e-9470-8701c9a21e6f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.066214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b263c76-dd60-4a8e-9470-8701c9a21e6f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.066502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b263c76-dd60-4a8e-9470-8701c9a21e6f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.072057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 134/272] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:06.072057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b263c76-dd60-4a8e-9470-8701c9a21e6f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.113963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:06.120880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 139/273] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:06.122203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a094e-d906-4756-9e17-05889f9f856a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.122534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a094e-d906-4756-9e17-05889f9f856a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.127753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a094e-d906-4756-9e17-05889f9f856a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.145885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a84a094e-d906-4756-9e17-05889f9f856a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:06.147079 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a84a094e-d906-4756-9e17-05889f9f856a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:06.173061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:06.198617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b263c76-dd60-4a8e-9470-8701c9a21e6f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.218629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 140/274] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:06.220062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a094e-d906-4756-9e17-05889f9f856a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.240370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25fa94-142a-4298-bc0f-3d3f9cd35206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.243524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25fa94-142a-4298-bc0f-3d3f9cd35206 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.243524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25fa94-142a-4298-bc0f-3d3f9cd35206 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.243524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25fa94-142a-4298-bc0f-3d3f9cd35206 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.252089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25fa94-142a-4298-bc0f-3d3f9cd35206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-df25fa94-142a-4298-bc0f-3d3f9cd35206 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:06.260059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:06.260928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:06.261709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:06.263763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 135/275] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:06.265257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25fa94-142a-4298-bc0f-3d3f9cd35206 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.327356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a094e-d906-4756-9e17-05889f9f856a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.358049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:06.363692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f966ce-9270-4554-a7dd-0889b335f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.364361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f966ce-9270-4554-a7dd-0889b335f734 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.364650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f966ce-9270-4554-a7dd-0889b335f734 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.364932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f966ce-9270-4554-a7dd-0889b335f734 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.365772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f966ce-9270-4554-a7dd-0889b335f734 None admin] RBAC: Adding query filter params (role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b, user.id=885394bcef08445f91a33940725efb7a) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:06.366089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f966ce-9270-4554-a7dd-0889b335f734 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:06.367580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f966ce-9270-4554-a7dd-0889b335f734 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:06.368257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25fa94-142a-4298-bc0f-3d3f9cd35206 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.373828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 141/276] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/role_assignments?user.id=885394bcef08445f91a33940725efb7a&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:06.374226 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f966ce-9270-4554-a7dd-0889b335f734 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.405213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:06.417082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c9d1d-eeeb-4c9a-85fb-8b6418b09220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.419082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c9d1d-eeeb-4c9a-85fb-8b6418b09220 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.419594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c9d1d-eeeb-4c9a-85fb-8b6418b09220 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.420064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c9d1d-eeeb-4c9a-85fb-8b6418b09220 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.421540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c9d1d-eeeb-4c9a-85fb-8b6418b09220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:06.423972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c9d1d-eeeb-4c9a-85fb-8b6418b09220 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:06.426507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 136/277] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:06.428280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c9d1d-eeeb-4c9a-85fb-8b6418b09220 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.494069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f966ce-9270-4554-a7dd-0889b335f734 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.533221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e73174-fb37-4238-9b56-961b2c455c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.534076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e73174-fb37-4238-9b56-961b2c455c6b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.534308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e73174-fb37-4238-9b56-961b2c455c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.535622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e73174-fb37-4238-9b56-961b2c455c6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.549328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e73174-fb37-4238-9b56-961b2c455c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:06.556937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c9d1d-eeeb-4c9a-85fb-8b6418b09220 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-10e73174-fb37-4238-9b56-961b2c455c6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:06.557163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:06.558007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:06.558804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:06.563793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 142/278] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:06.564759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e73174-fb37-4238-9b56-961b2c455c6b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.600672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2917e61-c24c-402f-90f9-f522a5dc4493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.601274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2917e61-c24c-402f-90f9-f522a5dc4493 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.601560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2917e61-c24c-402f-90f9-f522a5dc4493 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.601868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2917e61-c24c-402f-90f9-f522a5dc4493 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.602593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2917e61-c24c-402f-90f9-f522a5dc4493 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:06.602906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2917e61-c24c-402f-90f9-f522a5dc4493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:06.604342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2917e61-c24c-402f-90f9-f522a5dc4493 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:06.614581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 137/279] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:06.616436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2917e61-c24c-402f-90f9-f522a5dc4493 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.648406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:06.672234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:06.688905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e73174-fb37-4238-9b56-961b2c455c6b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.726954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496058-274a-4469-a046-3195007d261d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.727667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496058-274a-4469-a046-3195007d261d None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.727954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496058-274a-4469-a046-3195007d261d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.728378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496058-274a-4469-a046-3195007d261d None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.729005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496058-274a-4469-a046-3195007d261d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:06.730469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496058-274a-4469-a046-3195007d261d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:06.734260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2917e61-c24c-402f-90f9-f522a5dc4493 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.752321 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 143/280] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:28:06 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:28:06.760265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496058-274a-4469-a046-3195007d261d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.772202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2a0d4-f09e-4d58-8a34-7c9e8d7e2fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.772897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2a0d4-f09e-4d58-8a34-7c9e8d7e2fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.773183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2a0d4-f09e-4d58-8a34-7c9e8d7e2fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.773461 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2a0d4-f09e-4d58-8a34-7c9e8d7e2fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.774231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2a0d4-f09e-4d58-8a34-7c9e8d7e2fa2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:06.774540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2a0d4-f09e-4d58-8a34-7c9e8d7e2fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:06.776082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2a0d4-f09e-4d58-8a34-7c9e8d7e2fa2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:06.785770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 138/281] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:06.791663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2a0d4-f09e-4d58-8a34-7c9e8d7e2fa2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.893156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:06.910536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496058-274a-4469-a046-3195007d261d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.919614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2a0d4-f09e-4d58-8a34-7c9e8d7e2fa2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:06.946936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f4bd8-da62-432f-b3fe-b9297e75c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.948186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f4bd8-da62-432f-b3fe-b9297e75c34d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.948475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f4bd8-da62-432f-b3fe-b9297e75c34d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.948756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f4bd8-da62-432f-b3fe-b9297e75c34d None admin] PATH_INFO: `/v3/users/98d4d890e76940f586a32114acecbbe1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.961650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:06.965828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f4bd8-da62-432f-b3fe-b9297e75c34d None admin] RBAC: Authorizing `identity:get_user(user_id=98d4d890e76940f586a32114acecbbe1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:06.967484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f4bd8-da62-432f-b3fe-b9297e75c34d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:06.967978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56c7a5-b7bb-4e90-955f-3fb14bc6ceb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:06.968874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56c7a5-b7bb-4e90-955f-3fb14bc6ceb7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:06.969153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56c7a5-b7bb-4e90-955f-3fb14bc6ceb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:06.969428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56c7a5-b7bb-4e90-955f-3fb14bc6ceb7 None admin] PATH_INFO: `/v3/users/63b6fed1fb584dd7ba5578662c298ffd` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:06.972469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56c7a5-b7bb-4e90-955f-3fb14bc6ceb7 None admin] RBAC: Authorizing `identity:get_user(user_id=63b6fed1fb584dd7ba5578662c298ffd)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:06.974025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56c7a5-b7bb-4e90-955f-3fb14bc6ceb7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:06.976783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 139/282] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/users/63b6fed1fb584dd7ba5578662c298ffd => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:06.976923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 144/283] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/users/98d4d890e76940f586a32114acecbbe1 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:06.977844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56c7a5-b7bb-4e90-955f-3fb14bc6ceb7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:06.980908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f4bd8-da62-432f-b3fe-b9297e75c34d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:07.113003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f4bd8-da62-432f-b3fe-b9297e75c34d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:07.113200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56c7a5-b7bb-4e90-955f-3fb14bc6ceb7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:07.153577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c8a93-3551-478b-b62a-22779b904b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:07.154169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c8a93-3551-478b-b62a-22779b904b63 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:07.154259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c8a93-3551-478b-b62a-22779b904b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:07.154511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c8a93-3551-478b-b62a-22779b904b63 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:07.155737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c8a93-3551-478b-b62a-22779b904b63 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:07.157220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c8a93-3551-478b-b62a-22779b904b63 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:07.159201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 145/284] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:07.162252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efd251-2606-4e0b-bff0-33c2e712df4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:07.162921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efd251-2606-4e0b-bff0-33c2e712df4b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:07.163204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efd251-2606-4e0b-bff0-33c2e712df4b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:07.164298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c8a93-3551-478b-b62a-22779b904b63 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:07.164651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efd251-2606-4e0b-bff0-33c2e712df4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:07.165483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efd251-2606-4e0b-bff0-33c2e712df4b None admin] RBAC: Adding query filter params (role.id=0428a514a20043c680400ebf6498c74e, user.id=98d4d890e76940f586a32114acecbbe1, scope.system=all) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:07.165798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efd251-2606-4e0b-bff0-33c2e712df4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:07.167239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efd251-2606-4e0b-bff0-33c2e712df4b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:07.174512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 140/285] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Jul 13 20:28:06 2022] GET /identity/v3/role_assignments?user.id=98d4d890e76940f586a32114acecbbe1&scope.system=all&role.id=0428a514a20043c680400ebf6498c74e => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:07.254879 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:07.294915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c8a93-3551-478b-b62a-22779b904b63 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:07.334988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df958d06-24aa-426d-85f3-5733173cd247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:07.335948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958d06-24aa-426d-85f3-5733173cd247 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:07.336232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958d06-24aa-426d-85f3-5733173cd247 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:07.336509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958d06-24aa-426d-85f3-5733173cd247 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/users/63b6fed1fb584dd7ba5578662c298ffd/roles/b27dca438b9d4942b4739e98f7796831` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:07.344848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df958d06-24aa-426d-85f3-5733173cd247 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1cafe4585e6448983aef6e3d236bd3b, user_id=63b6fed1fb584dd7ba5578662c298ffd, role_id=b27dca438b9d4942b4739e98f7796831)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:07.345517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df958d06-24aa-426d-85f3-5733173cd247 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:07.359956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 146/286] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:28:07 2022] PUT /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/users/63b6fed1fb584dd7ba5578662c298ffd/roles/b27dca438b9d4942b4739e98f7796831 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:28:07.975639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062fd47-789d-41e2-9cc0-c948c2d99356 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:07.976336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062fd47-789d-41e2-9cc0-c948c2d99356 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:07.976824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062fd47-789d-41e2-9cc0-c948c2d99356 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:07.978651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 141/287] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:07.984383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570e2ad-ab54-40f6-832e-447d26d9e07c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:07.984891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570e2ad-ab54-40f6-832e-447d26d9e07c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:07.985450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570e2ad-ab54-40f6-832e-447d26d9e07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:08.004142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f570e2ad-ab54-40f6-832e-447d26d9e07c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:08.005621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f570e2ad-ab54-40f6-832e-447d26d9e07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:08.083794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 147/288] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:08.107655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062fd47-789d-41e2-9cc0-c948c2d99356 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:08.130950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:08.249228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062fd47-789d-41e2-9cc0-c948c2d99356 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:08.282590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaebbe-4ade-4673-89f2-213f6b8e7829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:08.283843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaebbe-4ade-4673-89f2-213f6b8e7829 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:08.284330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaebbe-4ade-4673-89f2-213f6b8e7829 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:08.284793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaebbe-4ade-4673-89f2-213f6b8e7829 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:08.305764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaebbe-4ade-4673-89f2-213f6b8e7829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-afaaebbe-4ade-4673-89f2-213f6b8e7829 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:08.313890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:08.314754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:08.315580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:08.318179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 142/289] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:08.324585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570e2ad-ab54-40f6-832e-447d26d9e07c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:08.415846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:08.477168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570e2ad-ab54-40f6-832e-447d26d9e07c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:08.522382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0cdab-96c8-487e-a7c7-281d33fc1fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:08.523824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0cdab-96c8-487e-a7c7-281d33fc1fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:08.524120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0cdab-96c8-487e-a7c7-281d33fc1fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:08.524409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0cdab-96c8-487e-a7c7-281d33fc1fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:08.525178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0cdab-96c8-487e-a7c7-281d33fc1fa5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:08.525492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0cdab-96c8-487e-a7c7-281d33fc1fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:08.526886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0cdab-96c8-487e-a7c7-281d33fc1fa5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:08.536303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 148/290] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:08.542005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaebbe-4ade-4673-89f2-213f6b8e7829 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:08.639719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:08.676708 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaebbe-4ade-4673-89f2-213f6b8e7829 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:08.724238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c05ee-21d0-4215-8984-bd8560857a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:08.725142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c05ee-21d0-4215-8984-bd8560857a41 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:08.725431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c05ee-21d0-4215-8984-bd8560857a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:08.725714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c05ee-21d0-4215-8984-bd8560857a41 None admin] PATH_INFO: `/v3/users/482df145b8c14524aa96fdfbd4ceb794` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:08.726977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c05ee-21d0-4215-8984-bd8560857a41 None admin] RBAC: Authorizing `identity:get_user(user_id=482df145b8c14524aa96fdfbd4ceb794)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:08.728479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c05ee-21d0-4215-8984-bd8560857a41 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:08.730306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 143/291] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:08 2022] GET /identity/v3/users/482df145b8c14524aa96fdfbd4ceb794 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:08.736213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0cdab-96c8-487e-a7c7-281d33fc1fa5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:08.840942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0cdab-96c8-487e-a7c7-281d33fc1fa5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:08.863831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:08.875473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b98a9-1e4d-4d16-87f3-1f73969b97f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:08.876157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b98a9-1e4d-4d16-87f3-1f73969b97f1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:08.876793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b98a9-1e4d-4d16-87f3-1f73969b97f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:08.877085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b98a9-1e4d-4d16-87f3-1f73969b97f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:08.949885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b98a9-1e4d-4d16-87f3-1f73969b97f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:08.951444 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b98a9-1e4d-4d16-87f3-1f73969b97f1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:09.050486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 149/292] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:09.055967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c05ee-21d0-4215-8984-bd8560857a41 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:09.077277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce0a14-9863-4146-84f3-313939082e8c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.077568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce0a14-9863-4146-84f3-313939082e8c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.077855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce0a14-9863-4146-84f3-313939082e8c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.079248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 150/293] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:09.084096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504b5b3-bcd2-47b7-a348-5789b56f6382 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.084447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504b5b3-bcd2-47b7-a348-5789b56f6382 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.084737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504b5b3-bcd2-47b7-a348-5789b56f6382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.101739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0504b5b3-bcd2-47b7-a348-5789b56f6382 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:09.102903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0504b5b3-bcd2-47b7-a348-5789b56f6382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:09.117073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:09.201419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 151/294] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:09.224109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504b5b3-bcd2-47b7-a348-5789b56f6382 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:09.234010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c05ee-21d0-4215-8984-bd8560857a41 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:09.304186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937c986-8aac-4001-a1e7-94a107c69b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:09.305082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937c986-8aac-4001-a1e7-94a107c69b46 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.305367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937c986-8aac-4001-a1e7-94a107c69b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.305641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937c986-8aac-4001-a1e7-94a107c69b46 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.312261 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937c986-8aac-4001-a1e7-94a107c69b46 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:09.313752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937c986-8aac-4001-a1e7-94a107c69b46 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:09.315550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 144/295] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:09 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:09.332506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937c986-8aac-4001-a1e7-94a107c69b46 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:09.348944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:09.351608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504b5b3-bcd2-47b7-a348-5789b56f6382 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:09.420336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029d869-0038-46b8-8d41-7147a6190d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:09.421071 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029d869-0038-46b8-8d41-7147a6190d4c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.421357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029d869-0038-46b8-8d41-7147a6190d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.421632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029d869-0038-46b8-8d41-7147a6190d4c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.443817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029d869-0038-46b8-8d41-7147a6190d4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1029d869-0038-46b8-8d41-7147a6190d4c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:09.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:09.455989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:09.456756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:09.468269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 152/296] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:09.470324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029d869-0038-46b8-8d41-7147a6190d4c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:09.490887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:09.523012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937c986-8aac-4001-a1e7-94a107c69b46 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:09.586176 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df18207-6568-4e4f-a3a2-27c79e5b2f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:09.586846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df18207-6568-4e4f-a3a2-27c79e5b2f5c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.587134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df18207-6568-4e4f-a3a2-27c79e5b2f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.587414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df18207-6568-4e4f-a3a2-27c79e5b2f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.588303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df18207-6568-4e4f-a3a2-27c79e5b2f5c None admin] RBAC: Adding query filter params (role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b, user.id=482df145b8c14524aa96fdfbd4ceb794) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:09.588622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df18207-6568-4e4f-a3a2-27c79e5b2f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:09.590091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df18207-6568-4e4f-a3a2-27c79e5b2f5c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:09.602513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 145/297] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:09 2022] GET /identity/v3/role_assignments?user.id=482df145b8c14524aa96fdfbd4ceb794&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=0428a514a20043c680400ebf6498c74e => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:09.646905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029d869-0038-46b8-8d41-7147a6190d4c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:09.700604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b8c11-de48-4dd7-8e35-5656599685a0 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.700921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b8c11-de48-4dd7-8e35-5656599685a0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.701079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b8c11-de48-4dd7-8e35-5656599685a0 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.703489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 146/298] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:09.708734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d732e-73a9-47e2-888c-2eabe05ccb5b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.709075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d732e-73a9-47e2-888c-2eabe05ccb5b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.709313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d732e-73a9-47e2-888c-2eabe05ccb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.736079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c49d732e-73a9-47e2-888c-2eabe05ccb5b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:09.737332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c49d732e-73a9-47e2-888c-2eabe05ccb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:09.740780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee558f51-54d0-4c21-9541-7e42d99e8f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:09.741532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558f51-54d0-4c21-9541-7e42d99e8f72 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.741823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558f51-54d0-4c21-9541-7e42d99e8f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.742104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558f51-54d0-4c21-9541-7e42d99e8f72 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.742877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee558f51-54d0-4c21-9541-7e42d99e8f72 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:09.743188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee558f51-54d0-4c21-9541-7e42d99e8f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:09.744652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee558f51-54d0-4c21-9541-7e42d99e8f72 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:09.762463 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 153/299] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:09.770098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee558f51-54d0-4c21-9541-7e42d99e8f72 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:09.855122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 147/300] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:09.856729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc95ca-7b98-418d-95a1-e76834ebab8a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.857142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc95ca-7b98-418d-95a1-e76834ebab8a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.857460 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc95ca-7b98-418d-95a1-e76834ebab8a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.858791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 148/301] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:09.864103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c6777-7d33-4522-ab98-843ffba0e8c2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.864449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c6777-7d33-4522-ab98-843ffba0e8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.864796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c6777-7d33-4522-ab98-843ffba0e8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.871752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:09.874428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:09.883928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2a5c6777-7d33-4522-ab98-843ffba0e8c2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:09.885164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2a5c6777-7d33-4522-ab98-843ffba0e8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:09.904319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee558f51-54d0-4c21-9541-7e42d99e8f72 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:09.948782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075c352-0f8c-4f8b-b8e7-ce120a2ccc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:09.951244 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075c352-0f8c-4f8b-b8e7-ce120a2ccc8e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:09.951793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075c352-0f8c-4f8b-b8e7-ce120a2ccc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:09.952514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075c352-0f8c-4f8b-b8e7-ce120a2ccc8e None admin] PATH_INFO: `/v3/users/1fe7c3ebce604c26921951af1d009b7a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:09.972520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075c352-0f8c-4f8b-b8e7-ce120a2ccc8e None admin] RBAC: Authorizing `identity:get_user(user_id=1fe7c3ebce604c26921951af1d009b7a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:09.973742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 149/302] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:09.974599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c6777-7d33-4522-ab98-843ffba0e8c2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:09.974718 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075c352-0f8c-4f8b-b8e7-ce120a2ccc8e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:09.977256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 154/303] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:09 2022] GET /identity/v3/users/1fe7c3ebce604c26921951af1d009b7a => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:09.984688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075c352-0f8c-4f8b-b8e7-ce120a2ccc8e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:10.081414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c6777-7d33-4522-ab98-843ffba0e8c2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.087584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075c352-0f8c-4f8b-b8e7-ce120a2ccc8e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.113791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:10.115141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd042fc-4c52-408a-984f-b0af460b431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:10.116470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd042fc-4c52-408a-984f-b0af460b431c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.116808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd042fc-4c52-408a-984f-b0af460b431c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.117088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd042fc-4c52-408a-984f-b0af460b431c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.118262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd042fc-4c52-408a-984f-b0af460b431c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:10.119763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd042fc-4c52-408a-984f-b0af460b431c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:10.121396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 150/304] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:10.121692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f5759-04a3-4467-92ec-f2eb2231df34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:10.122159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd042fc-4c52-408a-984f-b0af460b431c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:10.122408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5759-04a3-4467-92ec-f2eb2231df34 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.122772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5759-04a3-4467-92ec-f2eb2231df34 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.123057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5759-04a3-4467-92ec-f2eb2231df34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.123928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f5759-04a3-4467-92ec-f2eb2231df34 None admin] RBAC: Adding query filter params (role.id=b27dca438b9d4942b4739e98f7796831, user.id=1fe7c3ebce604c26921951af1d009b7a, scope.system=all) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:10.124247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f5759-04a3-4467-92ec-f2eb2231df34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:10.124787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:10.125684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f5759-04a3-4467-92ec-f2eb2231df34 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:10.143497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 155/305] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Jul 13 20:28:09 2022] GET /identity/v3/role_assignments?user.id=1fe7c3ebce604c26921951af1d009b7a&scope.system=all&role.id=b27dca438b9d4942b4739e98f7796831 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:10.143497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361d0b4-7ed7-41ac-96af-f4b9b8fa44f0 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.143497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361d0b4-7ed7-41ac-96af-f4b9b8fa44f0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.143497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361d0b4-7ed7-41ac-96af-f4b9b8fa44f0 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.143497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 156/306] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:10.143497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361d0b4-7ed7-41ac-96af-f4b9b8fa44f0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:10.240135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd042fc-4c52-408a-984f-b0af460b431c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.257637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361d0b4-7ed7-41ac-96af-f4b9b8fa44f0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.277122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7506a-af47-4a26-8763-d5795ebacb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:10.277934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7506a-af47-4a26-8763-d5795ebacb7d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.278217 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7506a-af47-4a26-8763-d5795ebacb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.278559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7506a-af47-4a26-8763-d5795ebacb7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.291193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7506a-af47-4a26-8763-d5795ebacb7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:10.297175 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6da8-deb1-4267-9e69-f67f83caf16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:10.297847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6da8-deb1-4267-9e69-f67f83caf16e None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.298125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6da8-deb1-4267-9e69-f67f83caf16e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.298397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6da8-deb1-4267-9e69-f67f83caf16e None admin] PATH_INFO: `/v3/groups` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f1e7506a-af47-4a26-8763-d5795ebacb7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:10.299070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:10.299924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:10.300687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:10.301258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a6da8-deb1-4267-9e69-f67f83caf16e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:10.301324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 151/307] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:10.302027 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112c32c-4e88-464c-8c63-625f4dcdb231 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.302321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112c32c-4e88-464c-8c63-625f4dcdb231 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.302606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112c32c-4e88-464c-8c63-625f4dcdb231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.303815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a6da8-deb1-4267-9e69-f67f83caf16e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:10.319910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4112c32c-4e88-464c-8c63-625f4dcdb231 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:10.321105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4112c32c-4e88-464c-8c63-625f4dcdb231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:10.322132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 157/308] 10.222.0.47 () {60 vars in 1134 bytes} [Wed Jul 13 20:28:10 2022] POST /identity/v3/groups => generated 229 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:28:10.323326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6da8-deb1-4267-9e69-f67f83caf16e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:10.388749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:10.404249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:10.427606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 152/309] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:10.451122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6da8-deb1-4267-9e69-f67f83caf16e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.455087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112c32c-4e88-464c-8c63-625f4dcdb231 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:10.495823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219b943-6b41-41ad-a685-72e7537995da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:10.495823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219b943-6b41-41ad-a685-72e7537995da None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.495823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219b943-6b41-41ad-a685-72e7537995da None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.495823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219b943-6b41-41ad-a685-72e7537995da None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.496183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219b943-6b41-41ad-a685-72e7537995da None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:10.496183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219b943-6b41-41ad-a685-72e7537995da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:10.496183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219b943-6b41-41ad-a685-72e7537995da None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:10.504888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 158/310] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:10.510431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219b943-6b41-41ad-a685-72e7537995da None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:10.575827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112c32c-4e88-464c-8c63-625f4dcdb231 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.621184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9aebd-2765-4826-b5b0-ff996280f0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:10.622246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9aebd-2765-4826-b5b0-ff996280f0aa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.622851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9aebd-2765-4826-b5b0-ff996280f0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.623272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9aebd-2765-4826-b5b0-ff996280f0aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.625233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219b943-6b41-41ad-a685-72e7537995da None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.637303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9aebd-2765-4826-b5b0-ff996280f0aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a2a9aebd-2765-4826-b5b0-ff996280f0aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:10.647474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:10.648565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:10.649542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:10.650432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:10.650590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 153/311] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:10.655316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9aebd-2765-4826-b5b0-ff996280f0aa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:10.663283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d61a3c-9af6-43fe-8f7e-a9f4e6aaf35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:10.664197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d61a3c-9af6-43fe-8f7e-a9f4e6aaf35b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.664476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d61a3c-9af6-43fe-8f7e-a9f4e6aaf35b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.664749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d61a3c-9af6-43fe-8f7e-a9f4e6aaf35b None admin] PATH_INFO: `/v3/users/98d4d890e76940f586a32114acecbbe1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.665929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d61a3c-9af6-43fe-8f7e-a9f4e6aaf35b None admin] RBAC: Authorizing `identity:get_user(user_id=98d4d890e76940f586a32114acecbbe1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:10.667379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d61a3c-9af6-43fe-8f7e-a9f4e6aaf35b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:10.670557 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 159/312] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:10 2022] GET /identity/v3/users/98d4d890e76940f586a32114acecbbe1 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:10.676157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d61a3c-9af6-43fe-8f7e-a9f4e6aaf35b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:10.717278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:10.793772 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9aebd-2765-4826-b5b0-ff996280f0aa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.805605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d61a3c-9af6-43fe-8f7e-a9f4e6aaf35b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.829206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f5d3e-27b0-4754-93c1-2c4036a3c5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:10.830066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f5d3e-27b0-4754-93c1-2c4036a3c5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.830449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f5d3e-27b0-4754-93c1-2c4036a3c5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.830826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f5d3e-27b0-4754-93c1-2c4036a3c5e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.831724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f5d3e-27b0-4754-93c1-2c4036a3c5e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:10.832130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f5d3e-27b0-4754-93c1-2c4036a3c5e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:10.833851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f5d3e-27b0-4754-93c1-2c4036a3c5e9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:10.844646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 154/313] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:10.849834 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f5d3e-27b0-4754-93c1-2c4036a3c5e9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:10.853557 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f842a-4536-48c5-bc67-0039a3072ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:10.854449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f842a-4536-48c5-bc67-0039a3072ab9 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:10.854736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f842a-4536-48c5-bc67-0039a3072ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:10.855017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f842a-4536-48c5-bc67-0039a3072ab9 None admin] PATH_INFO: `/v3/system/users/98d4d890e76940f586a32114acecbbe1/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:10.856771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f842a-4536-48c5-bc67-0039a3072ab9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=98d4d890e76940f586a32114acecbbe1, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:10.858188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f842a-4536-48c5-bc67-0039a3072ab9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:10.869524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 160/314] 10.222.0.47 () {58 vars in 1334 bytes} [Wed Jul 13 20:28:10 2022] PUT /identity/v3/system/users/98d4d890e76940f586a32114acecbbe1/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:10.975643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f5d3e-27b0-4754-93c1-2c4036a3c5e9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:10.991817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:11.016570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1a7f-842a-45ae-8c14-60493b53b4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:11.017830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1a7f-842a-45ae-8c14-60493b53b4ec None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:11.018324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1a7f-842a-45ae-8c14-60493b53b4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:11.018795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1a7f-842a-45ae-8c14-60493b53b4ec None admin] PATH_INFO: `/v3/users/63b6fed1fb584dd7ba5578662c298ffd` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:11.020685 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce1a7f-842a-45ae-8c14-60493b53b4ec None admin] RBAC: Authorizing `identity:get_user(user_id=63b6fed1fb584dd7ba5578662c298ffd)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:11.022470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce1a7f-842a-45ae-8c14-60493b53b4ec None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:11.027550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 155/315] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:10 2022] GET /identity/v3/users/63b6fed1fb584dd7ba5578662c298ffd => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:11.031057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f842a-4536-48c5-bc67-0039a3072ab9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:11.086116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:11.143251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f842a-4536-48c5-bc67-0039a3072ab9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:11.177866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088c3d-5f4b-4430-ad99-be2f5a0d8247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:11.178525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088c3d-5f4b-4430-ad99-be2f5a0d8247 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:11.178813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088c3d-5f4b-4430-ad99-be2f5a0d8247 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:11.179090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088c3d-5f4b-4430-ad99-be2f5a0d8247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:11.249789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed088c3d-5f4b-4430-ad99-be2f5a0d8247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:11.251291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed088c3d-5f4b-4430-ad99-be2f5a0d8247 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:11.298288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:11.352378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 161/316] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 20:28:11.360606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1a7f-842a-45ae-8c14-60493b53b4ec None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:11.465130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1a7f-842a-45ae-8c14-60493b53b4ec None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:11.499899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33faa5-f043-4d2e-8ab4-06f3448f1bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:11.500796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33faa5-f043-4d2e-8ab4-06f3448f1bda None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:11.501086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33faa5-f043-4d2e-8ab4-06f3448f1bda None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:11.501365 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33faa5-f043-4d2e-8ab4-06f3448f1bda None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:11.502609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33faa5-f043-4d2e-8ab4-06f3448f1bda None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:11.504139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33faa5-f043-4d2e-8ab4-06f3448f1bda None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:11.506344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 156/317] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:11 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:11.512489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088c3d-5f4b-4430-ad99-be2f5a0d8247 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:11.730718 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088c3d-5f4b-4430-ad99-be2f5a0d8247 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:11.813080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f48e7-8387-4dfa-ad27-2602612a24f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:11.813747 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f48e7-8387-4dfa-ad27-2602612a24f6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:11.814034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f48e7-8387-4dfa-ad27-2602612a24f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:11.814362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f48e7-8387-4dfa-ad27-2602612a24f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:11.815212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f48e7-8387-4dfa-ad27-2602612a24f6 None admin] RBAC: Adding query filter params (role.id=b27dca438b9d4942b4739e98f7796831, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b, user.id=63b6fed1fb584dd7ba5578662c298ffd) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:11.815572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f48e7-8387-4dfa-ad27-2602612a24f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:11.817089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f48e7-8387-4dfa-ad27-2602612a24f6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:11.823423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 162/318] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:11 2022] GET /identity/v3/role_assignments?user.id=63b6fed1fb584dd7ba5578662c298ffd&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=b27dca438b9d4942b4739e98f7796831 => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:12.177420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59fa03-97b0-428e-b261-21df8564d310 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.177800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59fa03-97b0-428e-b261-21df8564d310 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.177979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59fa03-97b0-428e-b261-21df8564d310 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.179642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 157/319] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:12.185123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3bfe1-e85b-4d15-b5e6-b4b908b1fd93 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.185421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3bfe1-e85b-4d15-b5e6-b4b908b1fd93 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.185708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3bfe1-e85b-4d15-b5e6-b4b908b1fd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.203533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e9c3bfe1-e85b-4d15-b5e6-b4b908b1fd93 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:12.204751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e9c3bfe1-e85b-4d15-b5e6-b4b908b1fd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:12.208674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:12.277529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 163/320] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:12.299606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59fa03-97b0-428e-b261-21df8564d310 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:12.357110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:12.406993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59fa03-97b0-428e-b261-21df8564d310 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:12.441136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b20b4-6dce-4107-b49d-0981073d7b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:12.441944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b20b4-6dce-4107-b49d-0981073d7b7f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.442225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b20b4-6dce-4107-b49d-0981073d7b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.442505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b20b4-6dce-4107-b49d-0981073d7b7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.453264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b20b4-6dce-4107-b49d-0981073d7b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ed8b20b4-6dce-4107-b49d-0981073d7b7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:12.460985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:12.461836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:12.467628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 158/321] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:12.469993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3bfe1-e85b-4d15-b5e6-b4b908b1fd93 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:12.574509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3bfe1-e85b-4d15-b5e6-b4b908b1fd93 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:12.587439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c16347-836d-4dc1-b7e9-f239f22e8e8e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.587855 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c16347-836d-4dc1-b7e9-f239f22e8e8e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.588204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c16347-836d-4dc1-b7e9-f239f22e8e8e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.589783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 159/322] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:12.595078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a9e88-2244-4d09-81d1-39931818109d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.595428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a9e88-2244-4d09-81d1-39931818109d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.595739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a9e88-2244-4d09-81d1-39931818109d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.612022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371afd6-83bf-4d65-b07f-cf990deb3087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:12.612635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371afd6-83bf-4d65-b07f-cf990deb3087 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.612922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371afd6-83bf-4d65-b07f-cf990deb3087 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.613203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371afd6-83bf-4d65-b07f-cf990deb3087 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.613301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-539a9e88-2244-4d09-81d1-39931818109d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:12.613960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371afd6-83bf-4d65-b07f-cf990deb3087 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:12.614273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371afd6-83bf-4d65-b07f-cf990deb3087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:12.614629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-539a9e88-2244-4d09-81d1-39931818109d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:12.615694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371afd6-83bf-4d65-b07f-cf990deb3087 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:12.624767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 164/323] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:12.630088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371afd6-83bf-4d65-b07f-cf990deb3087 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:12.632949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:12.686784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 160/324] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:12.709341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a9e88-2244-4d09-81d1-39931818109d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:12.744659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371afd6-83bf-4d65-b07f-cf990deb3087 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:12.752430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:12.781615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b126b6b-e8d1-459d-95bc-13357f37996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:12.782606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b126b6b-e8d1-459d-95bc-13357f37996e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.782893 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b126b6b-e8d1-459d-95bc-13357f37996e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.783176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b126b6b-e8d1-459d-95bc-13357f37996e None admin] PATH_INFO: `/v3/users/1fe7c3ebce604c26921951af1d009b7a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.784389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b126b6b-e8d1-459d-95bc-13357f37996e None admin] RBAC: Authorizing `identity:get_user(user_id=1fe7c3ebce604c26921951af1d009b7a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:12.785856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b126b6b-e8d1-459d-95bc-13357f37996e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:12.787824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 165/325] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:12 2022] GET /identity/v3/users/1fe7c3ebce604c26921951af1d009b7a => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:12.793021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b126b6b-e8d1-459d-95bc-13357f37996e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:12.819598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a9e88-2244-4d09-81d1-39931818109d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:12.854671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef548c1-40ec-460d-95d1-18d2beadf671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:12.855497 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef548c1-40ec-460d-95d1-18d2beadf671 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.855788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef548c1-40ec-460d-95d1-18d2beadf671 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.856066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef548c1-40ec-460d-95d1-18d2beadf671 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.857363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef548c1-40ec-460d-95d1-18d2beadf671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:12.858826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef548c1-40ec-460d-95d1-18d2beadf671 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:12.860639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 161/326] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:12.866143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef548c1-40ec-460d-95d1-18d2beadf671 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:12.867398 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:12.900919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b126b6b-e8d1-459d-95bc-13357f37996e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:12.936938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fe76a-49e8-49f0-be45-53f28844e90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:12.937808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fe76a-49e8-49f0-be45-53f28844e90d None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.938095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fe76a-49e8-49f0-be45-53f28844e90d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.938381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fe76a-49e8-49f0-be45-53f28844e90d None admin] PATH_INFO: `/v3/system/users/1fe7c3ebce604c26921951af1d009b7a/roles/b27dca438b9d4942b4739e98f7796831` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.940041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fe76a-49e8-49f0-be45-53f28844e90d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1fe7c3ebce604c26921951af1d009b7a, role_id=b27dca438b9d4942b4739e98f7796831)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:12.941513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fe76a-49e8-49f0-be45-53f28844e90d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:12.959505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 166/327] 10.222.0.47 () {58 vars in 1334 bytes} [Wed Jul 13 20:28:12 2022] PUT /identity/v3/system/users/1fe7c3ebce604c26921951af1d009b7a/roles/b27dca438b9d4942b4739e98f7796831 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:28:12.972273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbe3b4-03c9-48df-8a7e-43a83ffc5c94 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.972842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbe3b4-03c9-48df-8a7e-43a83ffc5c94 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.973132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbe3b4-03c9-48df-8a7e-43a83ffc5c94 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.974754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef548c1-40ec-460d-95d1-18d2beadf671 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:12.974871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 167/328] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:12.980600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aaa99-94a8-451f-9762-c0b1afaa92a4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:12.980930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aaa99-94a8-451f-9762-c0b1afaa92a4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:12.981340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aaa99-94a8-451f-9762-c0b1afaa92a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:12.989064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:12.999406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-963aaa99-94a8-451f-9762-c0b1afaa92a4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:13.000655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-963aaa99-94a8-451f-9762-c0b1afaa92a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:13.021091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb34b31-1d19-4a78-aaa7-225b55598be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:13.021767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb34b31-1d19-4a78-aaa7-225b55598be3 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:13.022056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb34b31-1d19-4a78-aaa7-225b55598be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:13.022335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb34b31-1d19-4a78-aaa7-225b55598be3 None admin] PATH_INFO: `/v3/groups` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:13.023043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb34b31-1d19-4a78-aaa7-225b55598be3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:13.024621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb34b31-1d19-4a78-aaa7-225b55598be3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:13.054342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 162/329] 10.222.0.47 () {60 vars in 1134 bytes} [Wed Jul 13 20:28:12 2022] POST /identity/v3/groups => generated 226 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:28:13.091545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 168/330] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:13.111668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb34b31-1d19-4a78-aaa7-225b55598be3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:13.221533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb34b31-1d19-4a78-aaa7-225b55598be3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:13.259237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4796a2b-0e6f-47d5-b0f9-e0beb011eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:13.260108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4796a2b-0e6f-47d5-b0f9-e0beb011eee1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:13.260394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4796a2b-0e6f-47d5-b0f9-e0beb011eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:13.260678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4796a2b-0e6f-47d5-b0f9-e0beb011eee1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:13.271214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4796a2b-0e6f-47d5-b0f9-e0beb011eee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b4796a2b-0e6f-47d5-b0f9-e0beb011eee1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:13.279113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:13.280031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:13.280903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:13.283635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 163/331] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:13.286777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aaa99-94a8-451f-9762-c0b1afaa92a4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:13.353667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:13.394989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aaa99-94a8-451f-9762-c0b1afaa92a4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:13.432803 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca08be-a6db-4611-9641-5caa4a43eb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:13.433488 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca08be-a6db-4611-9641-5caa4a43eb1e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:13.433776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca08be-a6db-4611-9641-5caa4a43eb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:13.434058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca08be-a6db-4611-9641-5caa4a43eb1e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:13.434882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca08be-a6db-4611-9641-5caa4a43eb1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:13.435192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca08be-a6db-4611-9641-5caa4a43eb1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:13.436849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca08be-a6db-4611-9641-5caa4a43eb1e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:13.447020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 169/332] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:13.452380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4796a2b-0e6f-47d5-b0f9-e0beb011eee1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:13.488463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:13.567709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4796a2b-0e6f-47d5-b0f9-e0beb011eee1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:13.605614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cdd42-7649-4433-8bde-c22c90980c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:13.606862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cdd42-7649-4433-8bde-c22c90980c2f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:13.607370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cdd42-7649-4433-8bde-c22c90980c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:13.607865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cdd42-7649-4433-8bde-c22c90980c2f None admin] PATH_INFO: `/v3/users/98d4d890e76940f586a32114acecbbe1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:13.609357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cdd42-7649-4433-8bde-c22c90980c2f None admin] RBAC: Authorizing `identity:get_user(user_id=98d4d890e76940f586a32114acecbbe1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:13.610994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cdd42-7649-4433-8bde-c22c90980c2f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:13.612837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 164/333] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:13 2022] GET /identity/v3/users/98d4d890e76940f586a32114acecbbe1 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:13.618778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca08be-a6db-4611-9641-5caa4a43eb1e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:13.751008 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca08be-a6db-4611-9641-5caa4a43eb1e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:13.784570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:13.793632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6dbd75-23b2-42d9-98e3-49fa95ca6ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:13.794442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dbd75-23b2-42d9-98e3-49fa95ca6ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:13.794727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dbd75-23b2-42d9-98e3-49fa95ca6ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:13.795002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dbd75-23b2-42d9-98e3-49fa95ca6ea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:13.795866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6dbd75-23b2-42d9-98e3-49fa95ca6ea5 None admin] RBAC: Adding query filter params (role.id=0428a514a20043c680400ebf6498c74e, user.id=98d4d890e76940f586a32114acecbbe1, scope.system=all) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:13.796173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6dbd75-23b2-42d9-98e3-49fa95ca6ea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:13.797640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6dbd75-23b2-42d9-98e3-49fa95ca6ea5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:13.812140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 170/334] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Jul 13 20:28:13 2022] GET /identity/v3/role_assignments?user.id=98d4d890e76940f586a32114acecbbe1&scope.system=all&role.id=0428a514a20043c680400ebf6498c74e => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:15.129159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a88034-cb4e-405d-abc8-50d68e50a40c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:15.129661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a88034-cb4e-405d-abc8-50d68e50a40c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:15.129762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a88034-cb4e-405d-abc8-50d68e50a40c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:15.131088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 165/335] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:15.136575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3e252-b16f-4b26-94e2-5a05ed5b094b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:15.136999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3e252-b16f-4b26-94e2-5a05ed5b094b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:15.137175 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3e252-b16f-4b26-94e2-5a05ed5b094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:15.164504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7cd3e252-b16f-4b26-94e2-5a05ed5b094b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:15.165711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7cd3e252-b16f-4b26-94e2-5a05ed5b094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:15.249297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 171/336] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:15.289048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a88034-cb4e-405d-abc8-50d68e50a40c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:15.402011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:15.410630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a88034-cb4e-405d-abc8-50d68e50a40c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:15.417301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e07a8-0b10-4052-a34b-a4eedcece34f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:15.417902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e07a8-0b10-4052-a34b-a4eedcece34f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:15.418370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e07a8-0b10-4052-a34b-a4eedcece34f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:15.424409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 172/337] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:15.429549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d860be-f0ca-4644-b9ce-a11d911983aa None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:15.430062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d860be-f0ca-4644-b9ce-a11d911983aa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:15.430639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d860be-f0ca-4644-b9ce-a11d911983aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:15.452047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49fa9f-415c-40d8-8908-a8b56cd755e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:15.452873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49fa9f-415c-40d8-8908-a8b56cd755e4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:15.453158 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49fa9f-415c-40d8-8908-a8b56cd755e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:15.453438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49fa9f-415c-40d8-8908-a8b56cd755e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:15.457329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d5d860be-f0ca-4644-b9ce-a11d911983aa None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:15.458713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d5d860be-f0ca-4644-b9ce-a11d911983aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:15.484997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49fa9f-415c-40d8-8908-a8b56cd755e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-cf49fa9f-415c-40d8-8908-a8b56cd755e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:15.497265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:15.498182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:15.498962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 166/338] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:15.518463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49fa9f-415c-40d8-8908-a8b56cd755e4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:15.588903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 173/339] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:15.669610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d860be-f0ca-4644-b9ce-a11d911983aa None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:15.685472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49fa9f-415c-40d8-8908-a8b56cd755e4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:15.739959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02d256-4c8c-4f85-9722-8c9ca6a8b584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:15.740642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02d256-4c8c-4f85-9722-8c9ca6a8b584 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:15.740929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02d256-4c8c-4f85-9722-8c9ca6a8b584 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:15.741209 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02d256-4c8c-4f85-9722-8c9ca6a8b584 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:15.747512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02d256-4c8c-4f85-9722-8c9ca6a8b584 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:15.747512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02d256-4c8c-4f85-9722-8c9ca6a8b584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:15.747512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02d256-4c8c-4f85-9722-8c9ca6a8b584 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:15.761291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 167/340] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:15.765883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226885a6-55b4-4a35-bd33-e73b52770dea None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:15.766230 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226885a6-55b4-4a35-bd33-e73b52770dea None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:15.766488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226885a6-55b4-4a35-bd33-e73b52770dea None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:15.767836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:15.768612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 168/341] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:15.772350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226885a6-55b4-4a35-bd33-e73b52770dea None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:15.806797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:15.885742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d860be-f0ca-4644-b9ce-a11d911983aa None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:15.944048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226885a6-55b4-4a35-bd33-e73b52770dea None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:15.968433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989b3ad-0129-422e-83b8-e2830512937a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:15.971046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989b3ad-0129-422e-83b8-e2830512937a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:15.971582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989b3ad-0129-422e-83b8-e2830512937a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:15.972806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989b3ad-0129-422e-83b8-e2830512937a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:15.997145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989b3ad-0129-422e-83b8-e2830512937a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-4989b3ad-0129-422e-83b8-e2830512937a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:16.005998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:16.006837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:16.007629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:16.018528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 174/342] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:16.019394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff31bc-4483-4c57-8295-5fb18b5fd3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:16.020543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff31bc-4483-4c57-8295-5fb18b5fd3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.021015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff31bc-4483-4c57-8295-5fb18b5fd3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.021475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff31bc-4483-4c57-8295-5fb18b5fd3c3 None admin] PATH_INFO: `/v3/users/1fe7c3ebce604c26921951af1d009b7a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.024194 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff31bc-4483-4c57-8295-5fb18b5fd3c3 None admin] RBAC: Authorizing `identity:get_user(user_id=1fe7c3ebce604c26921951af1d009b7a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:16.025842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff31bc-4483-4c57-8295-5fb18b5fd3c3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:16.026145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973872e-81a2-44fd-9f97-2a1d16a60837 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.026615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973872e-81a2-44fd-9f97-2a1d16a60837 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.027070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973872e-81a2-44fd-9f97-2a1d16a60837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.030214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 169/343] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:28:15 2022] GET /identity/v3/users/1fe7c3ebce604c26921951af1d009b7a => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:16.032393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff31bc-4483-4c57-8295-5fb18b5fd3c3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:16.051624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4973872e-81a2-44fd-9f97-2a1d16a60837 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:16.052860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4973872e-81a2-44fd-9f97-2a1d16a60837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:16.149469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:16.170624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff31bc-4483-4c57-8295-5fb18b5fd3c3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:16.189585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 175/344] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:16.190636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973872e-81a2-44fd-9f97-2a1d16a60837 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:16.210391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e80074-4219-4856-8466-d4a1805c3915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:16.211400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e80074-4219-4856-8466-d4a1805c3915 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.211917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e80074-4219-4856-8466-d4a1805c3915 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.212382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e80074-4219-4856-8466-d4a1805c3915 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.213359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e80074-4219-4856-8466-d4a1805c3915 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:16.213858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e80074-4219-4856-8466-d4a1805c3915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:16.215429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e80074-4219-4856-8466-d4a1805c3915 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:16.224784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 170/345] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:16.230200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e80074-4219-4856-8466-d4a1805c3915 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:16.237257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:16.349975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e80074-4219-4856-8466-d4a1805c3915 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:16.372387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973872e-81a2-44fd-9f97-2a1d16a60837 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:16.394578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e50a24-4ef1-4f8b-b125-d42141b8cc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:16.395242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e50a24-4ef1-4f8b-b125-d42141b8cc24 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.395567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e50a24-4ef1-4f8b-b125-d42141b8cc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.395859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e50a24-4ef1-4f8b-b125-d42141b8cc24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.418203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:16.437056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19e60a-d8b0-450f-bac5-4aa8baeaf144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:16.437695 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19e60a-d8b0-450f-bac5-4aa8baeaf144 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.437977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19e60a-d8b0-450f-bac5-4aa8baeaf144 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.438253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19e60a-d8b0-450f-bac5-4aa8baeaf144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.439555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19e60a-d8b0-450f-bac5-4aa8baeaf144 None admin] RBAC: Adding query filter params (role.id=b27dca438b9d4942b4739e98f7796831, user.id=1fe7c3ebce604c26921951af1d009b7a, scope.system=all) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:16.439555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19e60a-d8b0-450f-bac5-4aa8baeaf144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:16.440859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19e60a-d8b0-450f-bac5-4aa8baeaf144 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:16.462145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 176/346] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Jul 13 20:28:16 2022] GET /identity/v3/role_assignments?user.id=1fe7c3ebce604c26921951af1d009b7a&scope.system=all&role.id=b27dca438b9d4942b4739e98f7796831 => generated 510 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:16.462647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19e60a-d8b0-450f-bac5-4aa8baeaf144 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:16.476517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e50a24-4ef1-4f8b-b125-d42141b8cc24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:16.478432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e50a24-4ef1-4f8b-b125-d42141b8cc24 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:16.590161 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 171/347] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:16.596702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e50a24-4ef1-4f8b-b125-d42141b8cc24 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:16.616733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:16.640320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19e60a-d8b0-450f-bac5-4aa8baeaf144 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:16.666497 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:16.729442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e50a24-4ef1-4f8b-b125-d42141b8cc24 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:16.731426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb2081-af8a-4b2e-ab28-2ea887bac8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:16.732282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb2081-af8a-4b2e-ab28-2ea887bac8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.732566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb2081-af8a-4b2e-ab28-2ea887bac8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.732843 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb2081-af8a-4b2e-ab28-2ea887bac8f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.746910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb2081-af8a-4b2e-ab28-2ea887bac8f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bfbb2081-af8a-4b2e-ab28-2ea887bac8f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:16.756420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:16.757414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:16.758202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 177/348] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:28:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:16.761428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401b436-8461-48de-89d4-c27be1bce216 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.761850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401b436-8461-48de-89d4-c27be1bce216 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.762137 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401b436-8461-48de-89d4-c27be1bce216 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.763726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 178/349] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:16.767725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401b436-8461-48de-89d4-c27be1bce216 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:16.808556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f566b0-9ab5-4111-b6fd-43d8b24f4bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:16.810332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f566b0-9ab5-4111-b6fd-43d8b24f4bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.811431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f566b0-9ab5-4111-b6fd-43d8b24f4bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.812821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f566b0-9ab5-4111-b6fd-43d8b24f4bf6 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.815953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f566b0-9ab5-4111-b6fd-43d8b24f4bf6 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:16.818719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f566b0-9ab5-4111-b6fd-43d8b24f4bf6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:16.822474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 172/350] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:16 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:16.824596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ac93f-b16c-4878-aef7-1001c6f3295e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.825200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ac93f-b16c-4878-aef7-1001c6f3295e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.825673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ac93f-b16c-4878-aef7-1001c6f3295e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.848900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c52ac93f-b16c-4878-aef7-1001c6f3295e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:16.850103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c52ac93f-b16c-4878-aef7-1001c6f3295e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:16.910259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401b436-8461-48de-89d4-c27be1bce216 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:16.933772 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 173/351] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:16.934677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ac93f-b16c-4878-aef7-1001c6f3295e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:16.952899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58dff5-3010-4bad-b965-162520bc10e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:16.953918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58dff5-3010-4bad-b965-162520bc10e1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:16.954881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58dff5-3010-4bad-b965-162520bc10e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:16.955368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58dff5-3010-4bad-b965-162520bc10e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:16.956334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58dff5-3010-4bad-b965-162520bc10e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:16.956826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58dff5-3010-4bad-b965-162520bc10e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:16.958407 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58dff5-3010-4bad-b965-162520bc10e1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:16.969498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 179/352] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:16.975404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58dff5-3010-4bad-b965-162520bc10e1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:16.997449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:17.007100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:17.072529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ac93f-b16c-4878-aef7-1001c6f3295e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:17.114725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f65bc5-49a9-486f-9948-9975557275bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:17.115667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f65bc5-49a9-486f-9948-9975557275bc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.115954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f65bc5-49a9-486f-9948-9975557275bc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.116267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f65bc5-49a9-486f-9948-9975557275bc None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.122495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f65bc5-49a9-486f-9948-9975557275bc None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:17.124931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f65bc5-49a9-486f-9948-9975557275bc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:17.126980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 174/353] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:16 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:17.128234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58dff5-3010-4bad-b965-162520bc10e1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:17.132217 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f65bc5-49a9-486f-9948-9975557275bc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:17.171868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfe3c2-d8b3-4593-9a12-a4e666e336bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:17.173114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfe3c2-d8b3-4593-9a12-a4e666e336bc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.173533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfe3c2-d8b3-4593-9a12-a4e666e336bc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.173945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfe3c2-d8b3-4593-9a12-a4e666e336bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.187493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfe3c2-d8b3-4593-9a12-a4e666e336bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-dcdfe3c2-d8b3-4593-9a12-a4e666e336bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:17.200494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:17.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:17.203117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:17.205672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 180/354] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:17.207665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfe3c2-d8b3-4593-9a12-a4e666e336bc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:17.263893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f65bc5-49a9-486f-9948-9975557275bc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:17.279054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:17.292529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:17.312711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1a869-02e8-41a6-bbbd-6d16a246fc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:17.314560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1a869-02e8-41a6-bbbd-6d16a246fc42 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.314992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1a869-02e8-41a6-bbbd-6d16a246fc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.315426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1a869-02e8-41a6-bbbd-6d16a246fc42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.329319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfe3c2-d8b3-4593-9a12-a4e666e336bc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:17.380386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d205b9f-195b-4f6c-9039-7f01c8c2086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:17.391791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d205b9f-195b-4f6c-9039-7f01c8c2086c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.391791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d205b9f-195b-4f6c-9039-7f01c8c2086c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.391791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d205b9f-195b-4f6c-9039-7f01c8c2086c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.391791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d205b9f-195b-4f6c-9039-7f01c8c2086c None admin] RBAC: Adding query filter params (group.id=bf3fbb4b8cf6439a80796155cfe35fc5, role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=ab57ebe7c2f9472facbe45d50407af91) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:17.391791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d205b9f-195b-4f6c-9039-7f01c8c2086c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:17.391791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d205b9f-195b-4f6c-9039-7f01c8c2086c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:17.398095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 181/355] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:17 2022] GET /identity/v3/role_assignments?group.id=bf3fbb4b8cf6439a80796155cfe35fc5&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=0428a514a20043c680400ebf6498c74e => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:17.399395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d205b9f-195b-4f6c-9039-7f01c8c2086c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:17.412131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1a869-02e8-41a6-bbbd-6d16a246fc42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:17.415059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1a869-02e8-41a6-bbbd-6d16a246fc42 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:17.546879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 175/356] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:17.548423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d205b9f-195b-4f6c-9039-7f01c8c2086c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:17.548979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31380004-74f7-4d6a-ab81-d7bde44008e8 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.549470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31380004-74f7-4d6a-ab81-d7bde44008e8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.549939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31380004-74f7-4d6a-ab81-d7bde44008e8 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.552536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 176/357] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:17.553613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31380004-74f7-4d6a-ab81-d7bde44008e8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:17.577818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:17.600960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2012d-276d-420b-be58-f7dddd67557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:17.602020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2012d-276d-420b-be58-f7dddd67557d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.602521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2012d-276d-420b-be58-f7dddd67557d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.603002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2012d-276d-420b-be58-f7dddd67557d None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.604997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2012d-276d-420b-be58-f7dddd67557d None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:17.605537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2012d-276d-420b-be58-f7dddd67557d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:17.607207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2012d-276d-420b-be58-f7dddd67557d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:17.613111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:17.617700 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 182/358] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:17.619180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168c931-3f86-4b30-8497-888a977b0bd2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.619766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168c931-3f86-4b30-8497-888a977b0bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.621571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168c931-3f86-4b30-8497-888a977b0bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.640541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d168c931-3f86-4b30-8497-888a977b0bd2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:17.642022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d168c931-3f86-4b30-8497-888a977b0bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:17.679098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31380004-74f7-4d6a-ab81-d7bde44008e8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:17.716813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a758fb-1332-44a6-a381-bac310a27508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:17.717905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a758fb-1332-44a6-a381-bac310a27508 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.718202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a758fb-1332-44a6-a381-bac310a27508 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.718483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a758fb-1332-44a6-a381-bac310a27508 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.719830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a758fb-1332-44a6-a381-bac310a27508 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:17.720428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 183/359] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:17.721457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168c931-3f86-4b30-8497-888a977b0bd2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:17.730880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a758fb-1332-44a6-a381-bac310a27508 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:17.735035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 177/360] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:17 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:17.740514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a758fb-1332-44a6-a381-bac310a27508 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:17.825753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:17.850481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a758fb-1332-44a6-a381-bac310a27508 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:17.858831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168c931-3f86-4b30-8497-888a977b0bd2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:17.891415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:17.896494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747a378-d68f-468a-a40b-ca068f8eb5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:17.897377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747a378-d68f-468a-a40b-ca068f8eb5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.897662 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747a378-d68f-468a-a40b-ca068f8eb5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.897940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747a378-d68f-468a-a40b-ca068f8eb5e5 None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.899093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747a378-d68f-468a-a40b-ca068f8eb5e5 None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:17.899448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c60c0c-486a-4eb1-9d16-09c4f35c9ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:17.900147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c60c0c-486a-4eb1-9d16-09c4f35c9ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:17.900442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c60c0c-486a-4eb1-9d16-09c4f35c9ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:17.900597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747a378-d68f-468a-a40b-ca068f8eb5e5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:17.900758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c60c0c-486a-4eb1-9d16-09c4f35c9ff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:17.902394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 178/361] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:17 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:17.903102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747a378-d68f-468a-a40b-ca068f8eb5e5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:17.985274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c60c0c-486a-4eb1-9d16-09c4f35c9ff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:17.986775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c60c0c-486a-4eb1-9d16-09c4f35c9ff8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:18.025052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747a378-d68f-468a-a40b-ca068f8eb5e5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:18.077020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259a374-baaa-4062-b994-f74e4d3c2c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:18.078610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259a374-baaa-4062-b994-f74e4d3c2c86 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.078610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259a374-baaa-4062-b994-f74e4d3c2c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.078796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259a374-baaa-4062-b994-f74e4d3c2c86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.091845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259a374-baaa-4062-b994-f74e4d3c2c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c259a374-baaa-4062-b994-f74e4d3c2c86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:18.100141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:18.101068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:18.101888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 179/362] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:28:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:18.102677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ac086-e606-4896-a4c2-8455fc07241c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.102963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ac086-e606-4896-a4c2-8455fc07241c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.103500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ac086-e606-4896-a4c2-8455fc07241c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.104815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 180/363] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:18.105748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ac086-e606-4896-a4c2-8455fc07241c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:18.117564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:18.131247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 184/364] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:18.134249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c60c0c-486a-4eb1-9d16-09c4f35c9ff8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:18.170911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:18.236145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ac086-e606-4896-a4c2-8455fc07241c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:18.267126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c60c0c-486a-4eb1-9d16-09c4f35c9ff8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:18.281102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c08e4d-6e12-44ee-9f6e-f98fa8f98081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:18.281877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c08e4d-6e12-44ee-9f6e-f98fa8f98081 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.282162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c08e4d-6e12-44ee-9f6e-f98fa8f98081 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.282440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c08e4d-6e12-44ee-9f6e-f98fa8f98081 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.283317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c08e4d-6e12-44ee-9f6e-f98fa8f98081 None admin] RBAC: Adding query filter params (group.id=bf3fbb4b8cf6439a80796155cfe35fc5, role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=ab57ebe7c2f9472facbe45d50407af91) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:18.283717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c08e4d-6e12-44ee-9f6e-f98fa8f98081 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:18.285183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c08e4d-6e12-44ee-9f6e-f98fa8f98081 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:18.290980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 181/365] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:18 2022] GET /identity/v3/role_assignments?group.id=bf3fbb4b8cf6439a80796155cfe35fc5&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:18.292062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e231a-04b7-4a99-8b90-5fd582fec09a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.292351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e231a-04b7-4a99-8b90-5fd582fec09a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.292630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e231a-04b7-4a99-8b90-5fd582fec09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.310239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a605b3-1566-41f2-87ae-864d1d0f3508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:18.311396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a605b3-1566-41f2-87ae-864d1d0f3508 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.311922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a605b3-1566-41f2-87ae-864d1d0f3508 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.312107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6c4e231a-04b7-4a99-8b90-5fd582fec09a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:18.312593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a605b3-1566-41f2-87ae-864d1d0f3508 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.313403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6c4e231a-04b7-4a99-8b90-5fd582fec09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:18.313907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a605b3-1566-41f2-87ae-864d1d0f3508 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:18.314427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a605b3-1566-41f2-87ae-864d1d0f3508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:18.316128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a605b3-1566-41f2-87ae-864d1d0f3508 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:18.327228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 185/366] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:18.328459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a605b3-1566-41f2-87ae-864d1d0f3508 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:18.393880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:18.401174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 182/367] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:18.401174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e231a-04b7-4a99-8b90-5fd582fec09a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:18.464291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:18.475451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a605b3-1566-41f2-87ae-864d1d0f3508 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:18.513627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a3306-1bf3-47c5-87af-b27529731cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:18.514887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a3306-1bf3-47c5-87af-b27529731cbc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.515378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a3306-1bf3-47c5-87af-b27529731cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.515966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a3306-1bf3-47c5-87af-b27529731cbc None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.516379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e231a-04b7-4a99-8b90-5fd582fec09a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:18.517607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a3306-1bf3-47c5-87af-b27529731cbc None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:18.519432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a3306-1bf3-47c5-87af-b27529731cbc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:18.523223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 186/368] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:18 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:18.523223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a3306-1bf3-47c5-87af-b27529731cbc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:18.558098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0cb02-86ec-4ce1-aa8c-eb6155664e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:18.558816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0cb02-86ec-4ce1-aa8c-eb6155664e42 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.559107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0cb02-86ec-4ce1-aa8c-eb6155664e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.559421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0cb02-86ec-4ce1-aa8c-eb6155664e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.644909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:18.650124 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0cb02-86ec-4ce1-aa8c-eb6155664e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:18.650124 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0cb02-86ec-4ce1-aa8c-eb6155664e42 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:18.676234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a3306-1bf3-47c5-87af-b27529731cbc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:18.730827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ce50a-d089-42f7-b457-0582b1cec53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:18.732123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce50a-d089-42f7-b457-0582b1cec53c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.732608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce50a-d089-42f7-b457-0582b1cec53c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.733099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce50a-d089-42f7-b457-0582b1cec53c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.749609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ce50a-d089-42f7-b457-0582b1cec53c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9b0ce50a-d089-42f7-b457-0582b1cec53c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:18.760578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:18.761897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:18.762745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:18.765116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 187/369] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:18.766149 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ce50a-d089-42f7-b457-0582b1cec53c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:18.774094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 183/370] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:18.775093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0cb02-86ec-4ce1-aa8c-eb6155664e42 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:18.793492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:18.901883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:18.905224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0cb02-86ec-4ce1-aa8c-eb6155664e42 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:18.917558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ce50a-d089-42f7-b457-0582b1cec53c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:18.946109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b527e6-3cde-4859-9498-ad198cc0ff11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:18.946966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b527e6-3cde-4859-9498-ad198cc0ff11 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.947595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b527e6-3cde-4859-9498-ad198cc0ff11 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.948166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b527e6-3cde-4859-9498-ad198cc0ff11 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.950011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b527e6-3cde-4859-9498-ad198cc0ff11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:18.950704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b527e6-3cde-4859-9498-ad198cc0ff11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:18.954395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b527e6-3cde-4859-9498-ad198cc0ff11 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:18.962624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56a4c4-7ab0-4a93-9443-e01e5d9d28ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:18.963741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56a4c4-7ab0-4a93-9443-e01e5d9d28ac None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:18.964171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56a4c4-7ab0-4a93-9443-e01e5d9d28ac None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:18.964299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 184/371] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:18.964712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56a4c4-7ab0-4a93-9443-e01e5d9d28ac None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:18.965256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b527e6-3cde-4859-9498-ad198cc0ff11 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:18.966352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56a4c4-7ab0-4a93-9443-e01e5d9d28ac None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:18.968005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56a4c4-7ab0-4a93-9443-e01e5d9d28ac None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:18.970016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 188/372] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:18 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:18.971005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56a4c4-7ab0-4a93-9443-e01e5d9d28ac None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:19.100232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:19.104278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b527e6-3cde-4859-9498-ad198cc0ff11 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:19.113738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56a4c4-7ab0-4a93-9443-e01e5d9d28ac None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:19.151978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199da0d4-9059-4d6c-ad7a-eac12248528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:19.153006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199da0d4-9059-4d6c-ad7a-eac12248528d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:19.153369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199da0d4-9059-4d6c-ad7a-eac12248528d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:19.153651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199da0d4-9059-4d6c-ad7a-eac12248528d None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:19.156173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199da0d4-9059-4d6c-ad7a-eac12248528d None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:19.157654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199da0d4-9059-4d6c-ad7a-eac12248528d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:19.159767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 185/373] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:18 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:19.160556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199da0d4-9059-4d6c-ad7a-eac12248528d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:19.161026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a768-6f3f-4f2c-a570-914937a403f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:19.161766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a768-6f3f-4f2c-a570-914937a403f3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:19.162115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a768-6f3f-4f2c-a570-914937a403f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:19.162460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a768-6f3f-4f2c-a570-914937a403f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:19.199862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:19.242619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a768-6f3f-4f2c-a570-914937a403f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:19.244139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a768-6f3f-4f2c-a570-914937a403f3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:19.281213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199da0d4-9059-4d6c-ad7a-eac12248528d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:19.333764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f71f5e-082e-43d9-8a0f-7439bd6198ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:19.334755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f71f5e-082e-43d9-8a0f-7439bd6198ad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:19.335054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f71f5e-082e-43d9-8a0f-7439bd6198ad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:19.335331 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f71f5e-082e-43d9-8a0f-7439bd6198ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:19.336220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f71f5e-082e-43d9-8a0f-7439bd6198ad None admin] RBAC: Adding query filter params (group.id=bf3fbb4b8cf6439a80796155cfe35fc5, role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:19.336535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f71f5e-082e-43d9-8a0f-7439bd6198ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:19.338294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f71f5e-082e-43d9-8a0f-7439bd6198ad None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:19.348986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 186/374] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:19 2022] GET /identity/v3/role_assignments?group.id=bf3fbb4b8cf6439a80796155cfe35fc5&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=0428a514a20043c680400ebf6498c74e => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:19.349879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f71f5e-082e-43d9-8a0f-7439bd6198ad None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:19.372479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:19.392729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 189/375] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:19.398675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a768-6f3f-4f2c-a570-914937a403f3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:19.505345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:19.514892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f71f5e-082e-43d9-8a0f-7439bd6198ad None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:19.532868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a768-6f3f-4f2c-a570-914937a403f3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:19.589335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49843fd3-23fa-46e1-b6a9-1d060297a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:19.590216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49843fd3-23fa-46e1-b6a9-1d060297a03e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:19.590505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49843fd3-23fa-46e1-b6a9-1d060297a03e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:19.590812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49843fd3-23fa-46e1-b6a9-1d060297a03e None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:19.592049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49843fd3-23fa-46e1-b6a9-1d060297a03e None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:19.592719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdccc2-79e8-4b73-bc91-263cb0864c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:19.593486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49843fd3-23fa-46e1-b6a9-1d060297a03e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:19.593925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdccc2-79e8-4b73-bc91-263cb0864c4a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:19.594459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdccc2-79e8-4b73-bc91-263cb0864c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:19.595416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdccc2-79e8-4b73-bc91-263cb0864c4a None admin] PATH_INFO: `/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:19.597343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdccc2-79e8-4b73-bc91-263cb0864c4a None admin] RBAC: Authorizing `identity:get_project(project_id=81ec70650e114b9b8d48f50c1f3f4abe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:19.599730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 187/376] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:19 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:19.600229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdccc2-79e8-4b73-bc91-263cb0864c4a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:19.601644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49843fd3-23fa-46e1-b6a9-1d060297a03e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:19.603848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 190/377] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:19 2022] GET /identity/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:19.618256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdccc2-79e8-4b73-bc91-263cb0864c4a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:19.721608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49843fd3-23fa-46e1-b6a9-1d060297a03e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:19.743850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:19.759187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62075883-fc9f-42c4-a015-193cf561dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:19.759843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62075883-fc9f-42c4-a015-193cf561dadd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:19.760204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62075883-fc9f-42c4-a015-193cf561dadd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:19.760484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62075883-fc9f-42c4-a015-193cf561dadd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:19.761354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62075883-fc9f-42c4-a015-193cf561dadd None admin] RBAC: Adding query filter params (group.id=bf3fbb4b8cf6439a80796155cfe35fc5, role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:19.761660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62075883-fc9f-42c4-a015-193cf561dadd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:19.763293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62075883-fc9f-42c4-a015-193cf561dadd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:19.770278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 188/378] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:19 2022] GET /identity/v3/role_assignments?group.id=bf3fbb4b8cf6439a80796155cfe35fc5&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:19.778851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecd265-ac91-4325-8ddd-853c3d78e4d0 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:19.779193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecd265-ac91-4325-8ddd-853c3d78e4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:19.779405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecd265-ac91-4325-8ddd-853c3d78e4d0 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:19.780978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 189/379] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:19.786835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d58487-3ce8-436d-b1a5-1da2d329b4d2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:19.787139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d58487-3ce8-436d-b1a5-1da2d329b4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:19.787383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d58487-3ce8-436d-b1a5-1da2d329b4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:19.810177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdccc2-79e8-4b73-bc91-263cb0864c4a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:19.819993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-89d58487-3ce8-436d-b1a5-1da2d329b4d2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:19.821199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-89d58487-3ce8-436d-b1a5-1da2d329b4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:19.846987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f5a36-dba1-402f-b58a-320512de8696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:19.847918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f5a36-dba1-402f-b58a-320512de8696 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:19.848208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f5a36-dba1-402f-b58a-320512de8696 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:19.848493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f5a36-dba1-402f-b58a-320512de8696 None admin] PATH_INFO: `/v3/groups/21f23902ed3a4774826867224b62d203` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:19.857220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f5a36-dba1-402f-b58a-320512de8696 None admin] RBAC: Authorizing `identity:get_group(group_id=21f23902ed3a4774826867224b62d203)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:19.858710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f5a36-dba1-402f-b58a-320512de8696 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:19.861941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:19.863627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 191/380] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:19 2022] GET /identity/v3/groups/21f23902ed3a4774826867224b62d203 => generated 229 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:19.866869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f5a36-dba1-402f-b58a-320512de8696 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:19.955545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 190/381] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:19.995377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d58487-3ce8-436d-b1a5-1da2d329b4d2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:20.008336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f5a36-dba1-402f-b58a-320512de8696 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:20.048694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:20.118954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648da0-c9e6-4945-b74b-5537da929e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:20.119674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648da0-c9e6-4945-b74b-5537da929e1e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:20.119958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648da0-c9e6-4945-b74b-5537da929e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:20.120929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648da0-c9e6-4945-b74b-5537da929e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:20.121779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14648da0-c9e6-4945-b74b-5537da929e1e None admin] RBAC: Adding query filter params (group.id=21f23902ed3a4774826867224b62d203, role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=81ec70650e114b9b8d48f50c1f3f4abe) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:20.122200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14648da0-c9e6-4945-b74b-5537da929e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:20.124234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14648da0-c9e6-4945-b74b-5537da929e1e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:20.131378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 192/382] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:19 2022] GET /identity/v3/role_assignments?group.id=21f23902ed3a4774826867224b62d203&scope.project.id=81ec70650e114b9b8d48f50c1f3f4abe&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:20.172038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d58487-3ce8-436d-b1a5-1da2d329b4d2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:20.223029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13411c9c-946f-4610-90a1-0a0e0c5cede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:20.223872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13411c9c-946f-4610-90a1-0a0e0c5cede0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:20.224157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13411c9c-946f-4610-90a1-0a0e0c5cede0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:20.224434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13411c9c-946f-4610-90a1-0a0e0c5cede0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:20.238748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13411c9c-946f-4610-90a1-0a0e0c5cede0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-13411c9c-946f-4610-90a1-0a0e0c5cede0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:20.246561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:20.247538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:20.248338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:20.251312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 191/383] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:20.253914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648da0-c9e6-4945-b74b-5537da929e1e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:20.400909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648da0-c9e6-4945-b74b-5537da929e1e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:20.438503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476cc1d-17d2-43ce-9fe6-8cf39aecda5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:20.439188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476cc1d-17d2-43ce-9fe6-8cf39aecda5f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:20.439483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476cc1d-17d2-43ce-9fe6-8cf39aecda5f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:20.439780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476cc1d-17d2-43ce-9fe6-8cf39aecda5f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:20.440528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476cc1d-17d2-43ce-9fe6-8cf39aecda5f None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:20.440834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476cc1d-17d2-43ce-9fe6-8cf39aecda5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:20.442224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476cc1d-17d2-43ce-9fe6-8cf39aecda5f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:20.454808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 193/384] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:20.460311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13411c9c-946f-4610-90a1-0a0e0c5cede0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:20.482905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:20.579227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13411c9c-946f-4610-90a1-0a0e0c5cede0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:20.614152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc189a82-4e96-4a24-9c50-d55a072b03af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:20.615019 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc189a82-4e96-4a24-9c50-d55a072b03af None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:20.615302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc189a82-4e96-4a24-9c50-d55a072b03af None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:20.615606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc189a82-4e96-4a24-9c50-d55a072b03af None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:20.616849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc189a82-4e96-4a24-9c50-d55a072b03af None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:20.618317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc189a82-4e96-4a24-9c50-d55a072b03af None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:20.620328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 192/385] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:20 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:20.628167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476cc1d-17d2-43ce-9fe6-8cf39aecda5f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:20.681067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:20.690656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17235874-2706-4422-a8b1-6ae3b8363e88 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:20.690968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17235874-2706-4422-a8b1-6ae3b8363e88 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:20.691219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17235874-2706-4422-a8b1-6ae3b8363e88 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:20.695991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 193/386] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:20.699371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4da3d-623b-4300-8589-15415e493cf8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:20.699697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4da3d-623b-4300-8589-15415e493cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:20.700020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4da3d-623b-4300-8589-15415e493cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:20.720793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-36e4da3d-623b-4300-8589-15415e493cf8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:20.721950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-36e4da3d-623b-4300-8589-15415e493cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:20.754218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476cc1d-17d2-43ce-9fe6-8cf39aecda5f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:20.804168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042615e-0a39-4e8b-b4c4-260a759c57c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:20.805067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042615e-0a39-4e8b-b4c4-260a759c57c0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:20.805353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042615e-0a39-4e8b-b4c4-260a759c57c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:20.805629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042615e-0a39-4e8b-b4c4-260a759c57c0 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:20.807155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042615e-0a39-4e8b-b4c4-260a759c57c0 None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:20.810647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042615e-0a39-4e8b-b4c4-260a759c57c0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:20.812748 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 194/387] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:20 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:20.819852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042615e-0a39-4e8b-b4c4-260a759c57c0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:20.833442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 194/388] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:20.872840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4da3d-623b-4300-8589-15415e493cf8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:20.900069 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:20.978870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042615e-0a39-4e8b-b4c4-260a759c57c0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:20.994030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:21.028159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b80da-1c2d-44c1-b0a2-5b4b67a24c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:21.029111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b80da-1c2d-44c1-b0a2-5b4b67a24c94 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:21.029404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b80da-1c2d-44c1-b0a2-5b4b67a24c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:21.029685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b80da-1c2d-44c1-b0a2-5b4b67a24c94 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91/groups/bf3fbb4b8cf6439a80796155cfe35fc5/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:21.033468 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b80da-1c2d-44c1-b0a2-5b4b67a24c94 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab57ebe7c2f9472facbe45d50407af91, group_id=bf3fbb4b8cf6439a80796155cfe35fc5, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:21.034983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b80da-1c2d-44c1-b0a2-5b4b67a24c94 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:21.056656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 195/389] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 13 20:28:20 2022] PUT /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91/groups/bf3fbb4b8cf6439a80796155cfe35fc5/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:21.068114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4da3d-623b-4300-8589-15415e493cf8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:21.111709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f7650-0149-4503-b520-640a156c9197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:21.112604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f7650-0149-4503-b520-640a156c9197 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:21.112919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f7650-0149-4503-b520-640a156c9197 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:21.113211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f7650-0149-4503-b520-640a156c9197 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:21.127877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f7650-0149-4503-b520-640a156c9197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-764f7650-0149-4503-b520-640a156c9197 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:21.137183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:21.138191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:21.138944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 195/390] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:28:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:21.145006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b80da-1c2d-44c1-b0a2-5b4b67a24c94 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:21.261217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b80da-1c2d-44c1-b0a2-5b4b67a24c94 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:21.306065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085af57d-dbae-4ff9-b123-1324192dee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:21.306751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085af57d-dbae-4ff9-b123-1324192dee86 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:21.307036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085af57d-dbae-4ff9-b123-1324192dee86 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:21.307317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085af57d-dbae-4ff9-b123-1324192dee86 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:21.308107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085af57d-dbae-4ff9-b123-1324192dee86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:21.308419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085af57d-dbae-4ff9-b123-1324192dee86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:21.309874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085af57d-dbae-4ff9-b123-1324192dee86 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:21.321651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 196/391] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:21.327893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f7650-0149-4503-b520-640a156c9197 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:21.407016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:21.432557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f7650-0149-4503-b520-640a156c9197 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:21.468850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7b200-8175-4dfb-b322-3fb77e55a84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:21.469730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7b200-8175-4dfb-b322-3fb77e55a84d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:21.470016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7b200-8175-4dfb-b322-3fb77e55a84d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:21.470296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7b200-8175-4dfb-b322-3fb77e55a84d None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:21.471582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7b200-8175-4dfb-b322-3fb77e55a84d None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:21.473215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7b200-8175-4dfb-b322-3fb77e55a84d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:21.475186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 196/392] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:21 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:21.480786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085af57d-dbae-4ff9-b123-1324192dee86 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:21.518281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:21.604397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085af57d-dbae-4ff9-b123-1324192dee86 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:21.659446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9387c-633d-43a0-b902-98f55cfd842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:21.660701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9387c-633d-43a0-b902-98f55cfd842b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:21.661193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9387c-633d-43a0-b902-98f55cfd842b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:21.661657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9387c-633d-43a0-b902-98f55cfd842b None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:21.668200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9387c-633d-43a0-b902-98f55cfd842b None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:21.669940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9387c-633d-43a0-b902-98f55cfd842b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:21.672249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 197/393] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:21 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:21.677784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7b200-8175-4dfb-b322-3fb77e55a84d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:21.818401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7b200-8175-4dfb-b322-3fb77e55a84d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:21.865584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd58eec-2207-426f-b69c-0f1936394876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:21.866531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd58eec-2207-426f-b69c-0f1936394876 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:21.866825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd58eec-2207-426f-b69c-0f1936394876 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:21.867108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd58eec-2207-426f-b69c-0f1936394876 None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91/groups/bf3fbb4b8cf6439a80796155cfe35fc5/roles/32bea78dbfe74b56bd5c1f113586c690` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:21.869339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd58eec-2207-426f-b69c-0f1936394876 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab57ebe7c2f9472facbe45d50407af91, group_id=bf3fbb4b8cf6439a80796155cfe35fc5, role_id=32bea78dbfe74b56bd5c1f113586c690)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:21.870796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd58eec-2207-426f-b69c-0f1936394876 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:21.897645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 197/394] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 13 20:28:21 2022] PUT /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91/groups/bf3fbb4b8cf6439a80796155cfe35fc5/roles/32bea78dbfe74b56bd5c1f113586c690 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:22.098289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567988-54a9-4c16-8f35-5eec63e852a1 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:22.098899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567988-54a9-4c16-8f35-5eec63e852a1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:22.099381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567988-54a9-4c16-8f35-5eec63e852a1 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:22.101017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 198/395] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:22.116460 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872f1f3-7c10-4e57-bde7-589cf278915c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:22.116816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872f1f3-7c10-4e57-bde7-589cf278915c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:22.117104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872f1f3-7c10-4e57-bde7-589cf278915c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:22.122568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:22.138576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f872f1f3-7c10-4e57-bde7-589cf278915c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:22.140707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f872f1f3-7c10-4e57-bde7-589cf278915c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:22.224434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 198/396] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:22.247429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6567988-54a9-4c16-8f35-5eec63e852a1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:22.391877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6567988-54a9-4c16-8f35-5eec63e852a1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:22.400558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:22.444184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57b95b-fdf0-4252-82d3-f3bb720a72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:22.445345 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57b95b-fdf0-4252-82d3-f3bb720a72d0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:22.445853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57b95b-fdf0-4252-82d3-f3bb720a72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:22.446349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57b95b-fdf0-4252-82d3-f3bb720a72d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:22.461854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57b95b-fdf0-4252-82d3-f3bb720a72d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7f57b95b-fdf0-4252-82d3-f3bb720a72d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:22.469981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:22.470849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:22.471651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:22.474309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 199/397] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:22.480303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872f1f3-7c10-4e57-bde7-589cf278915c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:22.663938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872f1f3-7c10-4e57-bde7-589cf278915c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:22.699212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:22.705388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa2e5e-545e-4651-ab87-ea696f6db062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:22.706081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa2e5e-545e-4651-ab87-ea696f6db062 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:22.706368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa2e5e-545e-4651-ab87-ea696f6db062 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:22.706732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa2e5e-545e-4651-ab87-ea696f6db062 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:22.707517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa2e5e-545e-4651-ab87-ea696f6db062 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:22.707845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa2e5e-545e-4651-ab87-ea696f6db062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:22.709595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa2e5e-545e-4651-ab87-ea696f6db062 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:22.720226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 199/398] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:22.725647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57b95b-fdf0-4252-82d3-f3bb720a72d0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:22.808572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e3dfc-6b77-4fae-b671-219bc574dd23 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:22.808894 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e3dfc-6b77-4fae-b671-219bc574dd23 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:22.809148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e3dfc-6b77-4fae-b671-219bc574dd23 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:22.810803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 200/399] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:22.816112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764d064-4312-4dad-a65e-25b4a7461b83 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:22.816401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764d064-4312-4dad-a65e-25b4a7461b83 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:22.816734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764d064-4312-4dad-a65e-25b4a7461b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:22.851189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6764d064-4312-4dad-a65e-25b4a7461b83 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:22.853132 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6764d064-4312-4dad-a65e-25b4a7461b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:22.885268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57b95b-fdf0-4252-82d3-f3bb720a72d0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:22.957302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c278d-4e1e-4988-8ac3-89d9bf935109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:22.959196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c278d-4e1e-4988-8ac3-89d9bf935109 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:22.959494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c278d-4e1e-4988-8ac3-89d9bf935109 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:22.959774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c278d-4e1e-4988-8ac3-89d9bf935109 None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:22.961088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c278d-4e1e-4988-8ac3-89d9bf935109 None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:22.962536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c278d-4e1e-4988-8ac3-89d9bf935109 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:22.964186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 200/400] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:22 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:22.969764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc34652-0047-41d2-9e18-fa47d827ac8c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:22.970055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc34652-0047-41d2-9e18-fa47d827ac8c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:22.970343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc34652-0047-41d2-9e18-fa47d827ac8c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:22.971899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 201/401] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:22.972606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc34652-0047-41d2-9e18-fa47d827ac8c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:22.976223 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 201/402] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:22.983065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2257f55-7b06-4dde-a884-f2e6d49c69d5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:22.983352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2257f55-7b06-4dde-a884-f2e6d49c69d5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:22.983664 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2257f55-7b06-4dde-a884-f2e6d49c69d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.002038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e2257f55-7b06-4dde-a884-f2e6d49c69d5 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:23.003807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2257f55-7b06-4dde-a884-f2e6d49c69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:23.026664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:23.094891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 202/403] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:23.096191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2257f55-7b06-4dde-a884-f2e6d49c69d5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:23.130756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc34652-0047-41d2-9e18-fa47d827ac8c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:23.166988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:23.175141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3398b-4f4f-413a-a423-3b485240c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:23.176427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3398b-4f4f-413a-a423-3b485240c97c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.177144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3398b-4f4f-413a-a423-3b485240c97c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.177555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3398b-4f4f-413a-a423-3b485240c97c None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.179124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3398b-4f4f-413a-a423-3b485240c97c None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:23.180818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3398b-4f4f-413a-a423-3b485240c97c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:23.183781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 202/404] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:22 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:23.184944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3398b-4f4f-413a-a423-3b485240c97c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:23.226068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2257f55-7b06-4dde-a884-f2e6d49c69d5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:23.277354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb003f-1499-43f5-84e1-82443da333b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:23.279571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb003f-1499-43f5-84e1-82443da333b3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.279986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb003f-1499-43f5-84e1-82443da333b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.280366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb003f-1499-43f5-84e1-82443da333b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.296620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb003f-1499-43f5-84e1-82443da333b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:23.298981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:23.303101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3398b-4f4f-413a-a423-3b485240c97c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-39fb003f-1499-43f5-84e1-82443da333b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:23.310285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:23.311143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:23.312050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:23.315533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 203/405] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:28:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:23.316560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb003f-1499-43f5-84e1-82443da333b3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:23.349477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ceecf-44f2-4298-abd7-d9aa143caa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:23.350932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ceecf-44f2-4298-abd7-d9aa143caa9a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.351220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ceecf-44f2-4298-abd7-d9aa143caa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.351535 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ceecf-44f2-4298-abd7-d9aa143caa9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.365412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ceecf-44f2-4298-abd7-d9aa143caa9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-396ceecf-44f2-4298-abd7-d9aa143caa9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:23.376066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:23.377049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:23.377814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:23.378391 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 203/406] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:23.383084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ceecf-44f2-4298-abd7-d9aa143caa9a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:23.444987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb003f-1499-43f5-84e1-82443da333b3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:23.463586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:23.485394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fbac3-7ae6-42b6-9aab-9613a0a8b8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:23.486498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fbac3-7ae6-42b6-9aab-9613a0a8b8b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.486783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fbac3-7ae6-42b6-9aab-9613a0a8b8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.487062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fbac3-7ae6-42b6-9aab-9613a0a8b8b3 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/groups/bf3fbb4b8cf6439a80796155cfe35fc5/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.489481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fbac3-7ae6-42b6-9aab-9613a0a8b8b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1cafe4585e6448983aef6e3d236bd3b, group_id=bf3fbb4b8cf6439a80796155cfe35fc5, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:23.491212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fbac3-7ae6-42b6-9aab-9613a0a8b8b3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:23.504931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ceecf-44f2-4298-abd7-d9aa143caa9a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:23.516302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 204/407] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 13 20:28:23 2022] PUT /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/groups/bf3fbb4b8cf6439a80796155cfe35fc5/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:23.517356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fbac3-7ae6-42b6-9aab-9613a0a8b8b3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:23.572251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8c1fb-64d8-4bcd-9c5c-9c15a7ec61ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:23.572948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8c1fb-64d8-4bcd-9c5c-9c15a7ec61ac None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.573234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8c1fb-64d8-4bcd-9c5c-9c15a7ec61ac None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.573514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8c1fb-64d8-4bcd-9c5c-9c15a7ec61ac None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.574596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8c1fb-64d8-4bcd-9c5c-9c15a7ec61ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:23.574980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8c1fb-64d8-4bcd-9c5c-9c15a7ec61ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:23.578996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8c1fb-64d8-4bcd-9c5c-9c15a7ec61ac None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:23.601066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 204/408] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:23.607874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8c1fb-64d8-4bcd-9c5c-9c15a7ec61ac None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:23.658601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:23.705677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fbac3-7ae6-42b6-9aab-9613a0a8b8b3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:23.740740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cf2c7-345a-4eda-95a5-7a87096181a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:23.741337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cf2c7-345a-4eda-95a5-7a87096181a8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.741623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cf2c7-345a-4eda-95a5-7a87096181a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.741903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cf2c7-345a-4eda-95a5-7a87096181a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.742665 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cf2c7-345a-4eda-95a5-7a87096181a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:23.742977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cf2c7-345a-4eda-95a5-7a87096181a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:23.744426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cf2c7-345a-4eda-95a5-7a87096181a8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:23.754002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 205/409] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:23.759975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cf2c7-345a-4eda-95a5-7a87096181a8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:23.762716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8c1fb-64d8-4bcd-9c5c-9c15a7ec61ac None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:23.810977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:23.811918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db293487-9fe3-4ece-b124-52ac88b18b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:23.812856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db293487-9fe3-4ece-b124-52ac88b18b73 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.813141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db293487-9fe3-4ece-b124-52ac88b18b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.813423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db293487-9fe3-4ece-b124-52ac88b18b73 None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.814747 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db293487-9fe3-4ece-b124-52ac88b18b73 None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:23.817460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db293487-9fe3-4ece-b124-52ac88b18b73 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:23.819430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 205/410] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:23 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:23.820853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f05f1-58b2-45fa-aca7-013fc72454af None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.821143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f05f1-58b2-45fa-aca7-013fc72454af None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.822258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f05f1-58b2-45fa-aca7-013fc72454af None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.823549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 206/411] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:23.825233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f05f1-58b2-45fa-aca7-013fc72454af None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:23.894642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cf2c7-345a-4eda-95a5-7a87096181a8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:23.925634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:23.937187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f05f1-58b2-45fa-aca7-013fc72454af None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:23.943414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf80c8-6ab0-4502-97c8-c68f9f7e9c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:23.944396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf80c8-6ab0-4502-97c8-c68f9f7e9c0e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.944683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf80c8-6ab0-4502-97c8-c68f9f7e9c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.944956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf80c8-6ab0-4502-97c8-c68f9f7e9c0e None admin] PATH_INFO: `/v3/groups/21f23902ed3a4774826867224b62d203` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.946632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf80c8-6ab0-4502-97c8-c68f9f7e9c0e None admin] RBAC: Authorizing `identity:get_group(group_id=21f23902ed3a4774826867224b62d203)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:23.949078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf80c8-6ab0-4502-97c8-c68f9f7e9c0e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:23.951498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 206/412] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:23 2022] GET /identity/v3/groups/21f23902ed3a4774826867224b62d203 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:23.952755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a553b07-db6f-4928-a756-81cada3fb55c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.953046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a553b07-db6f-4928-a756-81cada3fb55c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.953326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a553b07-db6f-4928-a756-81cada3fb55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.976091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1a553b07-db6f-4928-a756-81cada3fb55c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:23.977456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1a553b07-db6f-4928-a756-81cada3fb55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:23.979132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da7772-1b4e-4691-9def-f92b17c2d986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:23.980258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da7772-1b4e-4691-9def-f92b17c2d986 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:23.980609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da7772-1b4e-4691-9def-f92b17c2d986 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:23.980955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da7772-1b4e-4691-9def-f92b17c2d986 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:23.982335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da7772-1b4e-4691-9def-f92b17c2d986 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:23.983905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da7772-1b4e-4691-9def-f92b17c2d986 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:23.985952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 207/413] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:23 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:23.986752 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da7772-1b4e-4691-9def-f92b17c2d986 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:24.070214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 207/414] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:24.071215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a553b07-db6f-4928-a756-81cada3fb55c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:24.119095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:24.125007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da7772-1b4e-4691-9def-f92b17c2d986 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:24.159781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f3262-5cf0-4c8a-8bb0-085b301be823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:24.160647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f3262-5cf0-4c8a-8bb0-085b301be823 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:24.160946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f3262-5cf0-4c8a-8bb0-085b301be823 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:24.161196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f3262-5cf0-4c8a-8bb0-085b301be823 None admin] PATH_INFO: `/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:24.162372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f3262-5cf0-4c8a-8bb0-085b301be823 None admin] RBAC: Authorizing `identity:get_project(project_id=81ec70650e114b9b8d48f50c1f3f4abe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:24.163852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f3262-5cf0-4c8a-8bb0-085b301be823 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:24.165860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 208/415] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:23 2022] GET /identity/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:24.166633 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f3262-5cf0-4c8a-8bb0-085b301be823 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:24.196500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:24.213602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a553b07-db6f-4928-a756-81cada3fb55c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:24.261901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19590f30-922b-4a99-888f-165c578f6a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:24.263870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19590f30-922b-4a99-888f-165c578f6a93 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:24.264380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19590f30-922b-4a99-888f-165c578f6a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:24.265101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19590f30-922b-4a99-888f-165c578f6a93 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/groups/bf3fbb4b8cf6439a80796155cfe35fc5/roles/32bea78dbfe74b56bd5c1f113586c690` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:24.268489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19590f30-922b-4a99-888f-165c578f6a93 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1cafe4585e6448983aef6e3d236bd3b, group_id=bf3fbb4b8cf6439a80796155cfe35fc5, role_id=32bea78dbfe74b56bd5c1f113586c690)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:24.271045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19590f30-922b-4a99-888f-165c578f6a93 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:24.272740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f3262-5cf0-4c8a-8bb0-085b301be823 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:24.291241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 208/416] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 13 20:28:24 2022] PUT /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b/groups/bf3fbb4b8cf6439a80796155cfe35fc5/roles/32bea78dbfe74b56bd5c1f113586c690 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:24.296453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19590f30-922b-4a99-888f-165c578f6a93 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:24.350990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31b791-83ae-4c90-8965-b513747c6f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:24.352299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31b791-83ae-4c90-8965-b513747c6f86 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:24.352800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31b791-83ae-4c90-8965-b513747c6f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:24.353258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31b791-83ae-4c90-8965-b513747c6f86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:24.376666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31b791-83ae-4c90-8965-b513747c6f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9b31b791-83ae-4c90-8965-b513747c6f86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:24.393360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:24.394289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:24.395083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:24.397906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 209/417] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:24.403733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31b791-83ae-4c90-8965-b513747c6f86 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:24.419946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:24.430192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19590f30-922b-4a99-888f-165c578f6a93 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:24.478191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa14e49-9659-4382-b862-30d5de2bec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:24.479934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa14e49-9659-4382-b862-30d5de2bec86 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:24.480856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa14e49-9659-4382-b862-30d5de2bec86 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:24.481598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa14e49-9659-4382-b862-30d5de2bec86 None admin] PATH_INFO: `/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe/groups/21f23902ed3a4774826867224b62d203/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:24.484645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa14e49-9659-4382-b862-30d5de2bec86 None admin] RBAC: Authorizing `identity:create_grant(project_id=81ec70650e114b9b8d48f50c1f3f4abe, group_id=21f23902ed3a4774826867224b62d203, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:24.487070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa14e49-9659-4382-b862-30d5de2bec86 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:24.506079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 209/418] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 13 20:28:24 2022] PUT /identity/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe/groups/21f23902ed3a4774826867224b62d203/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:24.512233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c7624-38bd-4777-9153-46bd639c74d1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:24.512233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c7624-38bd-4777-9153-46bd639c74d1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:24.512233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c7624-38bd-4777-9153-46bd639c74d1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:24.512233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 210/419] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:24.517963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258fe66-8848-4a67-842a-2de58cea7f66 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:24.520215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258fe66-8848-4a67-842a-2de58cea7f66 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:24.520495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258fe66-8848-4a67-842a-2de58cea7f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:24.527233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:24.541000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4258fe66-8848-4a67-842a-2de58cea7f66 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:24.542220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4258fe66-8848-4a67-842a-2de58cea7f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:24.553599 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31b791-83ae-4c90-8965-b513747c6f86 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:24.596944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355b83c-8f19-4231-882d-0dc7fe1e847f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:24.597645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355b83c-8f19-4231-882d-0dc7fe1e847f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:24.597929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355b83c-8f19-4231-882d-0dc7fe1e847f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:24.598203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355b83c-8f19-4231-882d-0dc7fe1e847f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:24.598967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355b83c-8f19-4231-882d-0dc7fe1e847f None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:24.599271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355b83c-8f19-4231-882d-0dc7fe1e847f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:24.600792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355b83c-8f19-4231-882d-0dc7fe1e847f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:24.611946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 210/420] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:24.617631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355b83c-8f19-4231-882d-0dc7fe1e847f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:24.635349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 211/421] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:24.667213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258fe66-8848-4a67-842a-2de58cea7f66 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:24.757832 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355b83c-8f19-4231-882d-0dc7fe1e847f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:24.782708 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:24.801859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8a170-cedc-41c2-a689-cd9aa1e7974c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:24.802519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8a170-cedc-41c2-a689-cd9aa1e7974c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:24.802814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8a170-cedc-41c2-a689-cd9aa1e7974c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:24.803090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8a170-cedc-41c2-a689-cd9aa1e7974c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:24.806738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258fe66-8848-4a67-842a-2de58cea7f66 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:24.836052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:24.876452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfbe95-a4ce-4e4c-8dcf-5fd274004dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:24.877195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfbe95-a4ce-4e4c-8dcf-5fd274004dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:24.877480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfbe95-a4ce-4e4c-8dcf-5fd274004dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:24.877762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfbe95-a4ce-4e4c-8dcf-5fd274004dc7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:24.898644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdfbe95-a4ce-4e4c-8dcf-5fd274004dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5cdfbe95-a4ce-4e4c-8dcf-5fd274004dc7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:24.909876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:24.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:24.911598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:24.915591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 212/422] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:28:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:24.919514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfbe95-a4ce-4e4c-8dcf-5fd274004dc7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:24.946785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8a170-cedc-41c2-a689-cd9aa1e7974c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:24.948526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8a170-cedc-41c2-a689-cd9aa1e7974c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:25.114248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfbe95-a4ce-4e4c-8dcf-5fd274004dc7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:25.116503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 211/423] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:25.117754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8a170-cedc-41c2-a689-cd9aa1e7974c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:25.169879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80a2d4-36d5-4c1d-85cf-2c3a94cc3ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:25.170468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80a2d4-36d5-4c1d-85cf-2c3a94cc3ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:25.170905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80a2d4-36d5-4c1d-85cf-2c3a94cc3ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:25.171181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80a2d4-36d5-4c1d-85cf-2c3a94cc3ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:25.172318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80a2d4-36d5-4c1d-85cf-2c3a94cc3ee0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:25.172626 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80a2d4-36d5-4c1d-85cf-2c3a94cc3ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:25.174024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80a2d4-36d5-4c1d-85cf-2c3a94cc3ee0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:25.181134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:25.182984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 213/424] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:25.192964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80a2d4-36d5-4c1d-85cf-2c3a94cc3ee0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:25.223852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:25.285653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8a170-cedc-41c2-a689-cd9aa1e7974c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:25.328270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56eedd5-bb71-4981-bf7b-9726a7d1226a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:25.329802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eedd5-bb71-4981-bf7b-9726a7d1226a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:25.330359 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eedd5-bb71-4981-bf7b-9726a7d1226a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:25.330640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eedd5-bb71-4981-bf7b-9726a7d1226a None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:25.332109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56eedd5-bb71-4981-bf7b-9726a7d1226a None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:25.333561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56eedd5-bb71-4981-bf7b-9726a7d1226a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:25.340591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 212/425] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:25 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:25.342243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56eedd5-bb71-4981-bf7b-9726a7d1226a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:25.403448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80a2d4-36d5-4c1d-85cf-2c3a94cc3ee0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:25.470113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdeb04-f852-4df3-81dc-64a9300d1bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:25.470769 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdeb04-f852-4df3-81dc-64a9300d1bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:25.471056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdeb04-f852-4df3-81dc-64a9300d1bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:25.471333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdeb04-f852-4df3-81dc-64a9300d1bb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:25.502361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56eedd5-bb71-4981-bf7b-9726a7d1226a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:25.521261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:25.544113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5af67-e4f9-482d-99fa-5ead94d5654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:25.545302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5af67-e4f9-482d-99fa-5ead94d5654a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:25.545781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5af67-e4f9-482d-99fa-5ead94d5654a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:25.546248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5af67-e4f9-482d-99fa-5ead94d5654a None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:25.547638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5af67-e4f9-482d-99fa-5ead94d5654a None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:25.550176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5af67-e4f9-482d-99fa-5ead94d5654a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:25.557358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 213/426] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:25 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:25.558758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5af67-e4f9-482d-99fa-5ead94d5654a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:25.621907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdeb04-f852-4df3-81dc-64a9300d1bb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:25.623395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdeb04-f852-4df3-81dc-64a9300d1bb2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:25.636528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:25.739963 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5af67-e4f9-482d-99fa-5ead94d5654a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:25.786151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 214/427] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:25.787517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c558089-385a-418a-b053-8a13e2db87c4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:25.787810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c558089-385a-418a-b053-8a13e2db87c4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:25.788086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c558089-385a-418a-b053-8a13e2db87c4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:25.790271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 215/428] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:25.793193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c558089-385a-418a-b053-8a13e2db87c4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:25.803230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfa7c4-c516-48ae-8774-9e06fe9ea92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:25.805432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfa7c4-c516-48ae-8774-9e06fe9ea92f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:25.805951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfa7c4-c516-48ae-8774-9e06fe9ea92f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:25.807057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfa7c4-c516-48ae-8774-9e06fe9ea92f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:25.808675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfa7c4-c516-48ae-8774-9e06fe9ea92f None admin] RBAC: Adding query filter params (group.id=bf3fbb4b8cf6439a80796155cfe35fc5, role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=ab57ebe7c2f9472facbe45d50407af91) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:25.809519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfa7c4-c516-48ae-8774-9e06fe9ea92f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:25.812009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfa7c4-c516-48ae-8774-9e06fe9ea92f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:25.827669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 214/429] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:25 2022] GET /identity/v3/role_assignments?group.id=bf3fbb4b8cf6439a80796155cfe35fc5&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=0428a514a20043c680400ebf6498c74e => generated 611 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:25.827669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e7ce5-f232-47d2-836c-41dfd5a023f4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:25.827669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e7ce5-f232-47d2-836c-41dfd5a023f4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:25.827669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e7ce5-f232-47d2-836c-41dfd5a023f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:25.863484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-478e7ce5-f232-47d2-836c-41dfd5a023f4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:25.864715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-478e7ce5-f232-47d2-836c-41dfd5a023f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:25.931795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:25.958957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 215/430] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:25.987969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c558089-385a-418a-b053-8a13e2db87c4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:25.999032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:26.013585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e7ce5-f232-47d2-836c-41dfd5a023f4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:26.032127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7309e4-0af8-4c5e-9306-14df7532890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:26.033668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7309e4-0af8-4c5e-9306-14df7532890c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.033959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7309e4-0af8-4c5e-9306-14df7532890c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.034237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7309e4-0af8-4c5e-9306-14df7532890c None admin] PATH_INFO: `/v3/projects/ab57ebe7c2f9472facbe45d50407af91` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.036444 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7309e4-0af8-4c5e-9306-14df7532890c None admin] RBAC: Authorizing `identity:get_project(project_id=ab57ebe7c2f9472facbe45d50407af91)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:26.038565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7309e4-0af8-4c5e-9306-14df7532890c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:26.040898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 216/431] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:25 2022] GET /identity/v3/projects/ab57ebe7c2f9472facbe45d50407af91 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:26.049187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7309e4-0af8-4c5e-9306-14df7532890c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:26.146626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e7ce5-f232-47d2-836c-41dfd5a023f4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:26.177324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7309e4-0af8-4c5e-9306-14df7532890c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:26.191386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aec984-eb88-4734-a21e-4f1b29358d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:26.192852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aec984-eb88-4734-a21e-4f1b29358d1d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.192852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aec984-eb88-4734-a21e-4f1b29358d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.193038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aec984-eb88-4734-a21e-4f1b29358d1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.206920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aec984-eb88-4734-a21e-4f1b29358d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-85aec984-eb88-4734-a21e-4f1b29358d1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:26.216933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:26.217811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:28:26.218608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:26.224351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 216/432] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:26.224983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aec984-eb88-4734-a21e-4f1b29358d1d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:26.228231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac86719-f9e0-45e5-8751-c7f328a3244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:26.230461 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac86719-f9e0-45e5-8751-c7f328a3244a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.231074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac86719-f9e0-45e5-8751-c7f328a3244a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.231691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac86719-f9e0-45e5-8751-c7f328a3244a None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.233061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac86719-f9e0-45e5-8751-c7f328a3244a None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:26.234874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac86719-f9e0-45e5-8751-c7f328a3244a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:26.238000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 217/433] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:26 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:26.246277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac86719-f9e0-45e5-8751-c7f328a3244a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:26.283192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:26.296758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:26.362720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac86719-f9e0-45e5-8751-c7f328a3244a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:26.372561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aec984-eb88-4734-a21e-4f1b29358d1d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:26.404010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d0d8d-ade8-4505-b988-ef18ed669850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:26.405066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d0d8d-ade8-4505-b988-ef18ed669850 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.405491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d0d8d-ade8-4505-b988-ef18ed669850 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.405891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d0d8d-ade8-4505-b988-ef18ed669850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.406860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d0d8d-ade8-4505-b988-ef18ed669850 None admin] RBAC: Adding query filter params (group.id=bf3fbb4b8cf6439a80796155cfe35fc5, role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=ab57ebe7c2f9472facbe45d50407af91) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:26.407379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d0d8d-ade8-4505-b988-ef18ed669850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:26.409126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d0d8d-ade8-4505-b988-ef18ed669850 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:26.413061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674e567-6f92-4eb8-bc43-027e4049e8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:26.413828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674e567-6f92-4eb8-bc43-027e4049e8ab None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.414218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674e567-6f92-4eb8-bc43-027e4049e8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.414594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674e567-6f92-4eb8-bc43-027e4049e8ab None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.415437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674e567-6f92-4eb8-bc43-027e4049e8ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:26.415871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674e567-6f92-4eb8-bc43-027e4049e8ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:26.417015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 218/434] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:26 2022] GET /identity/v3/role_assignments?group.id=bf3fbb4b8cf6439a80796155cfe35fc5&scope.project.id=ab57ebe7c2f9472facbe45d50407af91&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:26.423531 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674e567-6f92-4eb8-bc43-027e4049e8ab None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:26.427574 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 217/435] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:28:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:26.433140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d0d8d-ade8-4505-b988-ef18ed669850 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:26.548936 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d0d8d-ade8-4505-b988-ef18ed669850 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:26.557142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:26.588513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e863fb3-5bcd-4d8d-8b11-d343863cea97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:26.589443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e863fb3-5bcd-4d8d-8b11-d343863cea97 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.589863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e863fb3-5bcd-4d8d-8b11-d343863cea97 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.590269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e863fb3-5bcd-4d8d-8b11-d343863cea97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.659206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e863fb3-5bcd-4d8d-8b11-d343863cea97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:26.661357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e863fb3-5bcd-4d8d-8b11-d343863cea97 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:26.767697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 219/436] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:26.773461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674e567-6f92-4eb8-bc43-027e4049e8ab None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:26.782255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d629d7-da38-458a-9f16-b77ce69b099d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.782547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d629d7-da38-458a-9f16-b77ce69b099d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.782831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d629d7-da38-458a-9f16-b77ce69b099d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.784536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 220/437] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:26.790699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6682f6-be92-4a99-ab79-d772ef094149 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.790989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6682f6-be92-4a99-ab79-d772ef094149 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.791264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6682f6-be92-4a99-ab79-d772ef094149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.804663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:26.811439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6c6682f6-be92-4a99-ab79-d772ef094149 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:26.813339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6c6682f6-be92-4a99-ab79-d772ef094149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:26.889484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 221/438] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:26.891162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be316-532b-4341-a274-20ea69a67eec None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.891734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be316-532b-4341-a274-20ea69a67eec None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.892145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be316-532b-4341-a274-20ea69a67eec None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.893583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 222/439] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:26.898835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3510d4-59bc-45df-a597-6faa59b8386b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.899238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3510d4-59bc-45df-a597-6faa59b8386b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.899550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3510d4-59bc-45df-a597-6faa59b8386b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.907211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:26.910566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674e567-6f92-4eb8-bc43-027e4049e8ab None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:26.917345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0c3510d4-59bc-45df-a597-6faa59b8386b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:26.918581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0c3510d4-59bc-45df-a597-6faa59b8386b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:26.953247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58cb5-a93a-4c10-bf57-a8b4a915682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:26.954126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58cb5-a93a-4c10-bf57-a8b4a915682a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:26.954410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58cb5-a93a-4c10-bf57-a8b4a915682a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:26.954691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58cb5-a93a-4c10-bf57-a8b4a915682a None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:26.955932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc58cb5-a93a-4c10-bf57-a8b4a915682a None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:26.957391 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc58cb5-a93a-4c10-bf57-a8b4a915682a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:26.959100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 218/440] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:26 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:26.959884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58cb5-a93a-4c10-bf57-a8b4a915682a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:26.993409 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 223/441] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:26.994254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3510d4-59bc-45df-a597-6faa59b8386b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.058970 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:27.068570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58cb5-a93a-4c10-bf57-a8b4a915682a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.094871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3510d4-59bc-45df-a597-6faa59b8386b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.102749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a28ca6-336a-4cc6-a6af-44c94bf64f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.103580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a28ca6-336a-4cc6-a6af-44c94bf64f31 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.103864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a28ca6-336a-4cc6-a6af-44c94bf64f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.104136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a28ca6-336a-4cc6-a6af-44c94bf64f31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.114186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a28ca6-336a-4cc6-a6af-44c94bf64f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-40a28ca6-336a-4cc6-a6af-44c94bf64f31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:27.121858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:27.122861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:27.123741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 219/442] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:27.124408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a28ca6-336a-4cc6-a6af-44c94bf64f31 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.132891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c52cc2-c733-478a-8f18-fc045bec0a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.133753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c52cc2-c733-478a-8f18-fc045bec0a2f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.134033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c52cc2-c733-478a-8f18-fc045bec0a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.134303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c52cc2-c733-478a-8f18-fc045bec0a2f None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.135704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c52cc2-c733-478a-8f18-fc045bec0a2f None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:27.137205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c52cc2-c733-478a-8f18-fc045bec0a2f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:27.138911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 224/443] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:26 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:27.139689 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c52cc2-c733-478a-8f18-fc045bec0a2f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.153258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:27.224188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a28ca6-336a-4cc6-a6af-44c94bf64f31 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.235342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c52cc2-c733-478a-8f18-fc045bec0a2f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.257500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f8952-aa85-4706-9f36-b70975650179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.258331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f8952-aa85-4706-9f36-b70975650179 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.258621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f8952-aa85-4706-9f36-b70975650179 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.258901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f8952-aa85-4706-9f36-b70975650179 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.269063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f8952-aa85-4706-9f36-b70975650179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:27.270243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346dd25-8f9c-4c6b-ac0f-1f6af7b94738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.270911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346dd25-8f9c-4c6b-ac0f-1f6af7b94738 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.271193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346dd25-8f9c-4c6b-ac0f-1f6af7b94738 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.271477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346dd25-8f9c-4c6b-ac0f-1f6af7b94738 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.272227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346dd25-8f9c-4c6b-ac0f-1f6af7b94738 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:27.272536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346dd25-8f9c-4c6b-ac0f-1f6af7b94738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:27.273950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346dd25-8f9c-4c6b-ac0f-1f6af7b94738 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:27.275277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-841f8952-aa85-4706-9f36-b70975650179 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:27.277171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:27.278222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:27.278997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 220/444] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:27.279684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f8952-aa85-4706-9f36-b70975650179 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.283294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 225/445] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:27.288463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346dd25-8f9c-4c6b-ac0f-1f6af7b94738 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.372919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:27.384737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f8952-aa85-4706-9f36-b70975650179 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.385567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346dd25-8f9c-4c6b-ac0f-1f6af7b94738 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.417347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302607e8-c4df-4980-97b9-dd9347465c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.417999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302607e8-c4df-4980-97b9-dd9347465c75 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.418097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89f94f-da5b-4e2e-bb52-4cb754940029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.418281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302607e8-c4df-4980-97b9-dd9347465c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.418583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302607e8-c4df-4980-97b9-dd9347465c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.418748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89f94f-da5b-4e2e-bb52-4cb754940029 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.419037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89f94f-da5b-4e2e-bb52-4cb754940029 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.419315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89f94f-da5b-4e2e-bb52-4cb754940029 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.419409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302607e8-c4df-4980-97b9-dd9347465c75 None admin] RBAC: Adding query filter params (group.id=bf3fbb4b8cf6439a80796155cfe35fc5, role.id=0428a514a20043c680400ebf6498c74e, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:27.419733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302607e8-c4df-4980-97b9-dd9347465c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:27.420098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89f94f-da5b-4e2e-bb52-4cb754940029 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:27.420406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89f94f-da5b-4e2e-bb52-4cb754940029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:27.421198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302607e8-c4df-4980-97b9-dd9347465c75 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:27.421774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89f94f-da5b-4e2e-bb52-4cb754940029 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:27.427012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 221/446] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/role_assignments?group.id=bf3fbb4b8cf6439a80796155cfe35fc5&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=0428a514a20043c680400ebf6498c74e => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:27.427639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302607e8-c4df-4980-97b9-dd9347465c75 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.436791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 226/447] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:27.441362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89f94f-da5b-4e2e-bb52-4cb754940029 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.507532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:27.545291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302607e8-c4df-4980-97b9-dd9347465c75 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.555148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89f94f-da5b-4e2e-bb52-4cb754940029 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.582584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224721a-9e8a-4e00-b15b-5d423ea862f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.583248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224721a-9e8a-4e00-b15b-5d423ea862f5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.583577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224721a-9e8a-4e00-b15b-5d423ea862f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.583848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224721a-9e8a-4e00-b15b-5d423ea862f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.593391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54da52f-192e-49e5-9ebf-bf69aeadb1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.594052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54da52f-192e-49e5-9ebf-bf69aeadb1cb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.594341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54da52f-192e-49e5-9ebf-bf69aeadb1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.594622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54da52f-192e-49e5-9ebf-bf69aeadb1cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.615129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:27.671008 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224721a-9e8a-4e00-b15b-5d423ea862f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:27.672383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54da52f-192e-49e5-9ebf-bf69aeadb1cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:27.672796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224721a-9e8a-4e00-b15b-5d423ea862f5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:27.673162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54da52f-192e-49e5-9ebf-bf69aeadb1cb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:27.742856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:27.788723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 227/448] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 20:28:27.790844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 222/449] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:27.794514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54da52f-192e-49e5-9ebf-bf69aeadb1cb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.796229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224721a-9e8a-4e00-b15b-5d423ea862f5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.845879 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:27.888945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54da52f-192e-49e5-9ebf-bf69aeadb1cb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.891181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224721a-9e8a-4e00-b15b-5d423ea862f5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:27.922535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1684b-5a1b-46ff-913d-ef43f708bfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.923344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1684b-5a1b-46ff-913d-ef43f708bfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.923652 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1684b-5a1b-46ff-913d-ef43f708bfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.923928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1684b-5a1b-46ff-913d-ef43f708bfb5 None admin] PATH_INFO: `/v3/projects/d1cafe4585e6448983aef6e3d236bd3b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.924283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfac3bd-227c-45f3-a9e9-05f99b9590d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:27.925115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1684b-5a1b-46ff-913d-ef43f708bfb5 None admin] RBAC: Authorizing `identity:get_project(project_id=d1cafe4585e6448983aef6e3d236bd3b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:27.925209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfac3bd-227c-45f3-a9e9-05f99b9590d0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:27.925424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfac3bd-227c-45f3-a9e9-05f99b9590d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:27.925699 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfac3bd-227c-45f3-a9e9-05f99b9590d0 None admin] PATH_INFO: `/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:27.926568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1684b-5a1b-46ff-913d-ef43f708bfb5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:27.926917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfac3bd-227c-45f3-a9e9-05f99b9590d0 None admin] RBAC: Authorizing `identity:get_project(project_id=81ec70650e114b9b8d48f50c1f3f4abe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:27.928298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 228/450] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/projects/d1cafe4585e6448983aef6e3d236bd3b => generated 302 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:27.928431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfac3bd-227c-45f3-a9e9-05f99b9590d0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:27.930030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 223/451] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/projects/81ec70650e114b9b8d48f50c1f3f4abe => generated 344 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:27.933873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1684b-5a1b-46ff-913d-ef43f708bfb5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.935151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfac3bd-227c-45f3-a9e9-05f99b9590d0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:27.961746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:28.033371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1684b-5a1b-46ff-913d-ef43f708bfb5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:28.033773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfac3bd-227c-45f3-a9e9-05f99b9590d0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:28.053187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:28.066516 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eabdd-c8b1-4ccf-a66a-59b20473dc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:28.067391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eabdd-c8b1-4ccf-a66a-59b20473dc24 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:28.067701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eabdd-c8b1-4ccf-a66a-59b20473dc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:28.067982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eabdd-c8b1-4ccf-a66a-59b20473dc24 None admin] PATH_INFO: `/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:28.068483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57070b4-0dbc-4a6e-b404-96443f4d1595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:28.069149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516eabdd-c8b1-4ccf-a66a-59b20473dc24 None admin] RBAC: Authorizing `identity:get_group(group_id=bf3fbb4b8cf6439a80796155cfe35fc5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:28.069355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57070b4-0dbc-4a6e-b404-96443f4d1595 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:28.069648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57070b4-0dbc-4a6e-b404-96443f4d1595 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:28.069929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57070b4-0dbc-4a6e-b404-96443f4d1595 None admin] PATH_INFO: `/v3/groups/21f23902ed3a4774826867224b62d203` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:28.070601 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516eabdd-c8b1-4ccf-a66a-59b20473dc24 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:28.071126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57070b4-0dbc-4a6e-b404-96443f4d1595 None admin] RBAC: Authorizing `identity:get_group(group_id=21f23902ed3a4774826867224b62d203)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:28.072241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 229/452] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/groups/bf3fbb4b8cf6439a80796155cfe35fc5 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:28.072612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57070b4-0dbc-4a6e-b404-96443f4d1595 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:28.078952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 224/453] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 13 20:28:27 2022] GET /identity/v3/groups/21f23902ed3a4774826867224b62d203 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:28.079107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eabdd-c8b1-4ccf-a66a-59b20473dc24 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:28.080174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57070b4-0dbc-4a6e-b404-96443f4d1595 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:28.166646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:28.174063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eabdd-c8b1-4ccf-a66a-59b20473dc24 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:28.180438 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57070b4-0dbc-4a6e-b404-96443f4d1595 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:28.207675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f267bc-6f12-4496-b8c6-979c9f77285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:28.208331 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f267bc-6f12-4496-b8c6-979c9f77285a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:28.208619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f267bc-6f12-4496-b8c6-979c9f77285a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:28.208902 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f267bc-6f12-4496-b8c6-979c9f77285a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:28.209730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f267bc-6f12-4496-b8c6-979c9f77285a None admin] RBAC: Adding query filter params (group.id=bf3fbb4b8cf6439a80796155cfe35fc5, role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=d1cafe4585e6448983aef6e3d236bd3b) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:28.210044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f267bc-6f12-4496-b8c6-979c9f77285a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:28.211558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f267bc-6f12-4496-b8c6-979c9f77285a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:28.212539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518269f-89d5-4f62-950b-fc39921019ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:28.213191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518269f-89d5-4f62-950b-fc39921019ac None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:28.213477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518269f-89d5-4f62-950b-fc39921019ac None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:28.213752 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518269f-89d5-4f62-950b-fc39921019ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:28.214569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518269f-89d5-4f62-950b-fc39921019ac None admin] RBAC: Adding query filter params (group.id=21f23902ed3a4774826867224b62d203, role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=81ec70650e114b9b8d48f50c1f3f4abe) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:28.214881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518269f-89d5-4f62-950b-fc39921019ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:28.216384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518269f-89d5-4f62-950b-fc39921019ac None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:28.217563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 230/454] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:28 2022] GET /identity/v3/role_assignments?group.id=bf3fbb4b8cf6439a80796155cfe35fc5&scope.project.id=d1cafe4585e6448983aef6e3d236bd3b&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:28.224362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 225/455] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 13 20:28:28 2022] GET /identity/v3/role_assignments?group.id=21f23902ed3a4774826867224b62d203&scope.project.id=81ec70650e114b9b8d48f50c1f3f4abe&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:30.775744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef860b5-4bfe-4cf6-9485-7929e29fd526 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:30.776335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef860b5-4bfe-4cf6-9485-7929e29fd526 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:30.776335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef860b5-4bfe-4cf6-9485-7929e29fd526 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:30.777741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 231/456] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:30.783811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52f137-1157-4392-baa6-3a5f6ced9b74 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:30.784237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52f137-1157-4392-baa6-3a5f6ced9b74 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:30.784402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52f137-1157-4392-baa6-3a5f6ced9b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:30.807532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5f52f137-1157-4392-baa6-3a5f6ced9b74 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:30.808739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5f52f137-1157-4392-baa6-3a5f6ced9b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:30.871655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:30.899881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 226/457] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:30.936301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef860b5-4bfe-4cf6-9485-7929e29fd526 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:31.088140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef860b5-4bfe-4cf6-9485-7929e29fd526 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:31.134426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a975-148a-448d-9f66-2aeada134715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:31.135247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a975-148a-448d-9f66-2aeada134715 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.135555 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a975-148a-448d-9f66-2aeada134715 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.135838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a975-148a-448d-9f66-2aeada134715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.146369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003a975-148a-448d-9f66-2aeada134715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:31.147892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003a975-148a-448d-9f66-2aeada134715 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b003a975-148a-448d-9f66-2aeada134715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:31.174055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:31.174952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:31.175809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:31.175809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:31.175809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:31.175809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:31.175809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 232/458] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:31.181316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52f137-1157-4392-baa6-3a5f6ced9b74 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:31.219731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8a897-87c3-4684-acd2-2a5460442c4e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.220036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8a897-87c3-4684-acd2-2a5460442c4e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.220288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8a897-87c3-4684-acd2-2a5460442c4e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.221920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 233/459] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:31.236881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a3a16-47d9-40f1-be58-cf210bfd1ae1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.237168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a3a16-47d9-40f1-be58-cf210bfd1ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.237433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a3a16-47d9-40f1-be58-cf210bfd1ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.258649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-358a3a16-47d9-40f1-be58-cf210bfd1ae1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:31.259876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-358a3a16-47d9-40f1-be58-cf210bfd1ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:31.263752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:31.341744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52f137-1157-4392-baa6-3a5f6ced9b74 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:31.354802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 234/460] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:31.359753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a0a51-3c4f-4fb4-baaf-7a444443262e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.360061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a0a51-3c4f-4fb4-baaf-7a444443262e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.360343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a0a51-3c4f-4fb4-baaf-7a444443262e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.361915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 235/461] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:31.369216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8318f2-2e7d-4b20-9e01-53471b1130d8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.371078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8318f2-2e7d-4b20-9e01-53471b1130d8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.371525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8318f2-2e7d-4b20-9e01-53471b1130d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.394101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9e8318f2-2e7d-4b20-9e01-53471b1130d8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:31.395706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9e8318f2-2e7d-4b20-9e01-53471b1130d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:31.396576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a84665-58fc-41c5-8007-068dcab83997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:31.397252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a84665-58fc-41c5-8007-068dcab83997 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.397537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a84665-58fc-41c5-8007-068dcab83997 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.397815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a84665-58fc-41c5-8007-068dcab83997 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.398820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a84665-58fc-41c5-8007-068dcab83997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:31.399135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a84665-58fc-41c5-8007-068dcab83997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:31.401413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a84665-58fc-41c5-8007-068dcab83997 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:31.417100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 227/462] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:31.417947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a84665-58fc-41c5-8007-068dcab83997 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:31.468718 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:31.490534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 236/463] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:31.495532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8318f2-2e7d-4b20-9e01-53471b1130d8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:31.544763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a84665-58fc-41c5-8007-068dcab83997 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:31.566417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:31.581143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcff05-b4cc-4844-b5b2-851ad19eb67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:31.582038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcff05-b4cc-4844-b5b2-851ad19eb67f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.582233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcff05-b4cc-4844-b5b2-851ad19eb67f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.582506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcff05-b4cc-4844-b5b2-851ad19eb67f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.593933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcff05-b4cc-4844-b5b2-851ad19eb67f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:31.595324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcff05-b4cc-4844-b5b2-851ad19eb67f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:31.598519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8318f2-2e7d-4b20-9e01-53471b1130d8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-62fcff05-b4cc-4844-b5b2-851ad19eb67f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:31.612621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:31.613651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:31.614480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:31.614480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:31.614480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:31.614480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:31.614480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 228/464] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:31.615401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a559c-12ce-4e2d-b2a4-78f234ac126a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.615401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a559c-12ce-4e2d-b2a4-78f234ac126a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.615607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a559c-12ce-4e2d-b2a4-78f234ac126a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.619507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 229/465] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:31.619507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a559c-12ce-4e2d-b2a4-78f234ac126a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:31.639926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb94d00-68a0-4430-989f-ffbf1570e112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:31.640590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94d00-68a0-4430-989f-ffbf1570e112 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.640881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94d00-68a0-4430-989f-ffbf1570e112 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.641163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94d00-68a0-4430-989f-ffbf1570e112 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.641865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb94d00-68a0-4430-989f-ffbf1570e112 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:31.643342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb94d00-68a0-4430-989f-ffbf1570e112 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:31.663757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 237/466] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:28:31 2022] POST /identity/v3/users => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:28:31.668617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb94d00-68a0-4430-989f-ffbf1570e112 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:31.727787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:31.736310 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a559c-12ce-4e2d-b2a4-78f234ac126a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:31.778739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a69b90-706e-4553-80c8-a5eb197ab067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:31.779666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a69b90-706e-4553-80c8-a5eb197ab067 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.779951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a69b90-706e-4553-80c8-a5eb197ab067 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.781105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a69b90-706e-4553-80c8-a5eb197ab067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.792819 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a69b90-706e-4553-80c8-a5eb197ab067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:31.794658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a69b90-706e-4553-80c8-a5eb197ab067 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f7a69b90-706e-4553-80c8-a5eb197ab067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:31.814653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:31.815785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:31.816813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:31.816813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:31.816813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:31.816813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:31.822940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 230/467] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:31.822940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32383e-b7d4-4696-97c8-b0bedd6bb7ab None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.822940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32383e-b7d4-4696-97c8-b0bedd6bb7ab None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.822940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32383e-b7d4-4696-97c8-b0bedd6bb7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.831417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb94d00-68a0-4430-989f-ffbf1570e112 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:31.841474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2d32383e-b7d4-4696-97c8-b0bedd6bb7ab None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:31.842649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2d32383e-b7d4-4696-97c8-b0bedd6bb7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:31.863528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:31.878145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa515ca-05da-4651-b1e3-28c24603adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:31.879288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa515ca-05da-4651-b1e3-28c24603adfb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:31.879715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa515ca-05da-4651-b1e3-28c24603adfb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:31.880357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa515ca-05da-4651-b1e3-28c24603adfb None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:31.881201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa515ca-05da-4651-b1e3-28c24603adfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:31.881684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa515ca-05da-4651-b1e3-28c24603adfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:31.883723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa515ca-05da-4651-b1e3-28c24603adfb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:31.898781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 238/468] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:31.899726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa515ca-05da-4651-b1e3-28c24603adfb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:31.936522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 231/469] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:31.937615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32383e-b7d4-4696-97c8-b0bedd6bb7ab None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:32.021522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:32.035340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa515ca-05da-4651-b1e3-28c24603adfb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:32.046671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32383e-b7d4-4696-97c8-b0bedd6bb7ab None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:32.072691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50786fc-c3f2-4303-8002-572c2ca07b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:32.073359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50786fc-c3f2-4303-8002-572c2ca07b8d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.073950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50786fc-c3f2-4303-8002-572c2ca07b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.074225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50786fc-c3f2-4303-8002-572c2ca07b8d None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.074988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50786fc-c3f2-4303-8002-572c2ca07b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:32.075294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50786fc-c3f2-4303-8002-572c2ca07b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:32.076724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50786fc-c3f2-4303-8002-572c2ca07b8d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:32.088255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 239/470] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:32.089377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50786fc-c3f2-4303-8002-572c2ca07b8d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:32.093279 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759ed97-8bb6-4cad-99c4-6302f5b0ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:32.093969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759ed97-8bb6-4cad-99c4-6302f5b0ee21 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.094253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759ed97-8bb6-4cad-99c4-6302f5b0ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.094529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759ed97-8bb6-4cad-99c4-6302f5b0ee21 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.095241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759ed97-8bb6-4cad-99c4-6302f5b0ee21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:32.096742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759ed97-8bb6-4cad-99c4-6302f5b0ee21 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:32.119684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 232/471] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:28:31 2022] POST /identity/v3/users => generated 248 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:28:32.121413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e085e7d-42e2-450b-b971-ab44a19eeed6 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.121929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e085e7d-42e2-450b-b971-ab44a19eeed6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.122400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e085e7d-42e2-450b-b971-ab44a19eeed6 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.128470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 233/472] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:32.129731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e085e7d-42e2-450b-b971-ab44a19eeed6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:32.149749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:32.211736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50786fc-c3f2-4303-8002-572c2ca07b8d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:32.252644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9d57c-28f8-451c-bdea-d0e19768bdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:32.253612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9d57c-28f8-451c-bdea-d0e19768bdfb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.253899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9d57c-28f8-451c-bdea-d0e19768bdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.254225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9d57c-28f8-451c-bdea-d0e19768bdfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.265206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e085e7d-42e2-450b-b971-ab44a19eeed6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:32.266744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9d57c-28f8-451c-bdea-d0e19768bdfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:32.268664 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9d57c-28f8-451c-bdea-d0e19768bdfb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3aa9d57c-28f8-451c-bdea-d0e19768bdfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:32.288507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:32.289415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:32.290233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:32.290233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:32.290233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:32.290233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:32.290233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 240/473] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:32.291243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e9bef-27af-4159-968c-61ddb3656b6e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.291359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e9bef-27af-4159-968c-61ddb3656b6e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.291669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e9bef-27af-4159-968c-61ddb3656b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.313181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c00e9bef-27af-4159-968c-61ddb3656b6e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:32.313509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:32.314720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c00e9bef-27af-4159-968c-61ddb3656b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:32.320213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcc84a-3402-4869-b384-70e6318d19b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:32.320915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcc84a-3402-4869-b384-70e6318d19b5 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.321938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcc84a-3402-4869-b384-70e6318d19b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.322216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcc84a-3402-4869-b384-70e6318d19b5 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.322964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcc84a-3402-4869-b384-70e6318d19b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:32.324902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcc84a-3402-4869-b384-70e6318d19b5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:32.350827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 234/474] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:28:32 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:28:32.352071 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcc84a-3402-4869-b384-70e6318d19b5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:32.427142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 241/475] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:32.453715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e9bef-27af-4159-968c-61ddb3656b6e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:32.482852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcc84a-3402-4869-b384-70e6318d19b5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:32.492053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:32.533809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b06fe1-5736-4e22-bcc2-f7bd0124e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:32.534959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b06fe1-5736-4e22-bcc2-f7bd0124e75d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.535443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b06fe1-5736-4e22-bcc2-f7bd0124e75d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.536091 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b06fe1-5736-4e22-bcc2-f7bd0124e75d None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.537627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b06fe1-5736-4e22-bcc2-f7bd0124e75d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:32.538377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b06fe1-5736-4e22-bcc2-f7bd0124e75d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:32.540137 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b06fe1-5736-4e22-bcc2-f7bd0124e75d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:32.553611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 235/476] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:32.560254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b06fe1-5736-4e22-bcc2-f7bd0124e75d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:32.578571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e9bef-27af-4159-968c-61ddb3656b6e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:32.617473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:32.620298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:32.621214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.621812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.622311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.624364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:32.626467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:32.636191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.sql.core [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 20:28:32.636191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 20:28:32.636191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [parameters: {'id': '772fd00206ca467d8db3d86a8dce89f9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 20:28:32.636191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 13 20:28:32.659774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return ref.to_dict() Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:28:32.660857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application yield resource Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise self.value Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application session.commit() Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 20:28:32.661935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.flush() Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application compat.raise_( Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 20:28:32.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:28:32.664293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:28:32.665277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [parameters: {'id': '772fd00206ca467d8db3d86a8dce89f9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:32.666225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 20:28:32.667185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:32.676017 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 242/477] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 13 20:28:32 2022] POST /identity/v3/roles => generated 144 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:32.677653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:32.698432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b06fe1-5736-4e22-bcc2-f7bd0124e75d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:32.744673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f483df-c809-42db-a40e-f5b1ca4b7eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:32.745901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f483df-c809-42db-a40e-f5b1ca4b7eb3 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.746383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f483df-c809-42db-a40e-f5b1ca4b7eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.746985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f483df-c809-42db-a40e-f5b1ca4b7eb3 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.748216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f483df-c809-42db-a40e-f5b1ca4b7eb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:32.750060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f483df-c809-42db-a40e-f5b1ca4b7eb3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:32.778921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 236/478] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:28:32 2022] POST /identity/v3/users => generated 249 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:28:32.789897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:32.802580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee6fa6-f554-4bd8-983a-67e3cdf4210d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:32.849521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0003e-fb99-43fb-a0fc-9da0da51f3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:32.850459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0003e-fb99-43fb-a0fc-9da0da51f3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:32.850766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0003e-fb99-43fb-a0fc-9da0da51f3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:32.851254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0003e-fb99-43fb-a0fc-9da0da51f3b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:32.866476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0003e-fb99-43fb-a0fc-9da0da51f3b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4fb0003e-fb99-43fb-a0fc-9da0da51f3b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:32.875735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:32.876682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:32.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 243/479] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:28:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:32.884885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f483df-c809-42db-a40e-f5b1ca4b7eb3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:33.025406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f483df-c809-42db-a40e-f5b1ca4b7eb3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:33.028735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:33.069081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29030be8-47d6-4c16-998f-b03e0eeb08fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:33.069778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29030be8-47d6-4c16-998f-b03e0eeb08fc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:33.070063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29030be8-47d6-4c16-998f-b03e0eeb08fc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:33.070391 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29030be8-47d6-4c16-998f-b03e0eeb08fc None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:33.071213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29030be8-47d6-4c16-998f-b03e0eeb08fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:33.071564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29030be8-47d6-4c16-998f-b03e0eeb08fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:33.073016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29030be8-47d6-4c16-998f-b03e0eeb08fc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:33.083141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 237/480] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:34.114661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511812c-bc9a-4d50-b4e8-d2e4d158f45a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:34.115206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511812c-bc9a-4d50-b4e8-d2e4d158f45a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:34.115206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511812c-bc9a-4d50-b4e8-d2e4d158f45a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:34.116887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 244/481] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:34.128645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc2068-ab28-47d8-a322-3f6460cfc434 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:34.129058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc2068-ab28-47d8-a322-3f6460cfc434 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:34.129250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc2068-ab28-47d8-a322-3f6460cfc434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:34.149905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-00fc2068-ab28-47d8-a322-3f6460cfc434 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:34.151105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-00fc2068-ab28-47d8-a322-3f6460cfc434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:34.240118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 238/482] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:34.283157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511812c-bc9a-4d50-b4e8-d2e4d158f45a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:34.410181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511812c-bc9a-4d50-b4e8-d2e4d158f45a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:34.458821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc7f1d-aed2-413c-9d7e-23dfec4460f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:34.459664 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc7f1d-aed2-413c-9d7e-23dfec4460f3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:34.459954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc7f1d-aed2-413c-9d7e-23dfec4460f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:34.460238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc7f1d-aed2-413c-9d7e-23dfec4460f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:34.478610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc7f1d-aed2-413c-9d7e-23dfec4460f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b8bc7f1d-aed2-413c-9d7e-23dfec4460f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:34.486456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:34.487291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:34.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 245/483] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:34.493553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc2068-ab28-47d8-a322-3f6460cfc434 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:34.624757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:34.630902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc2068-ab28-47d8-a322-3f6460cfc434 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:34.678575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c44a5-62ab-4820-855d-54aa764de56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:34.679167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c44a5-62ab-4820-855d-54aa764de56f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:34.679445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c44a5-62ab-4820-855d-54aa764de56f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:34.679728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c44a5-62ab-4820-855d-54aa764de56f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:34.680480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c44a5-62ab-4820-855d-54aa764de56f None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:34.680778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c44a5-62ab-4820-855d-54aa764de56f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:34.682156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c44a5-62ab-4820-855d-54aa764de56f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:34.694651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 239/484] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:34.700632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc7f1d-aed2-413c-9d7e-23dfec4460f3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:34.705138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d36062-aed9-4354-af93-f4e2f17e3cd0 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:34.705426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d36062-aed9-4354-af93-f4e2f17e3cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:34.705705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d36062-aed9-4354-af93-f4e2f17e3cd0 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:34.707284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 240/485] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:34.716446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0baa96-a87f-4b19-a478-8ae2a82cab40 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:34.716712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0baa96-a87f-4b19-a478-8ae2a82cab40 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:34.716994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0baa96-a87f-4b19-a478-8ae2a82cab40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:34.733182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:34.748564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3d0baa96-a87f-4b19-a478-8ae2a82cab40 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:34.749863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3d0baa96-a87f-4b19-a478-8ae2a82cab40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:34.869275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 241/486] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:34.896215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0baa96-a87f-4b19-a478-8ae2a82cab40 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:34.898646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc7f1d-aed2-413c-9d7e-23dfec4460f3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:34.963007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d1675-93a6-4180-9826-cf8df7aaca84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:34.964007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d1675-93a6-4180-9826-cf8df7aaca84 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:34.964288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d1675-93a6-4180-9826-cf8df7aaca84 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:34.964561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d1675-93a6-4180-9826-cf8df7aaca84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:34.987218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d1675-93a6-4180-9826-cf8df7aaca84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:34.988725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d1675-93a6-4180-9826-cf8df7aaca84 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f34d1675-93a6-4180-9826-cf8df7aaca84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:35.017776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:35.018654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:35.019933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:35.019933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:35.019933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:35.019933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:35.019933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 246/487] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:35.031124 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d1675-93a6-4180-9826-cf8df7aaca84 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.031812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:35.059490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0baa96-a87f-4b19-a478-8ae2a82cab40 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.100745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:35.107928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d4af8-911f-4dca-9ea9-3a1b4fbd00fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.109013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d4af8-911f-4dca-9ea9-3a1b4fbd00fb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.109491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d4af8-911f-4dca-9ea9-3a1b4fbd00fb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.110012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d4af8-911f-4dca-9ea9-3a1b4fbd00fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.120899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d4af8-911f-4dca-9ea9-3a1b4fbd00fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-082d4af8-911f-4dca-9ea9-3a1b4fbd00fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:35.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:35.130424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:35.131177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:35.137036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 242/488] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:35.137036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95026322-7497-4113-b202-e630d0129557 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.137036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95026322-7497-4113-b202-e630d0129557 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.137036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95026322-7497-4113-b202-e630d0129557 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.137036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 243/489] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:35.140065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95026322-7497-4113-b202-e630d0129557 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.147991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d1675-93a6-4180-9826-cf8df7aaca84 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.184812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4676b52-662d-4608-8384-dc7b3becb7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.185914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4676b52-662d-4608-8384-dc7b3becb7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.186413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4676b52-662d-4608-8384-dc7b3becb7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.187103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4676b52-662d-4608-8384-dc7b3becb7d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.188384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4676b52-662d-4608-8384-dc7b3becb7d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:35.188869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4676b52-662d-4608-8384-dc7b3becb7d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:35.190691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4676b52-662d-4608-8384-dc7b3becb7d0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:35.208953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 247/490] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:35.210625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1071a-3e52-4775-9048-a7cb693450f0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.211108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1071a-3e52-4775-9048-a7cb693450f0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.211577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1071a-3e52-4775-9048-a7cb693450f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.229789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8af1071a-3e52-4775-9048-a7cb693450f0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:35.231393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8af1071a-3e52-4775-9048-a7cb693450f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:35.255960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95026322-7497-4113-b202-e630d0129557 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.292883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84042b03-f11d-4828-91c3-661277e1e510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.293952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84042b03-f11d-4828-91c3-661277e1e510 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.294379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84042b03-f11d-4828-91c3-661277e1e510 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.294798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84042b03-f11d-4828-91c3-661277e1e510 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.295823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84042b03-f11d-4828-91c3-661277e1e510 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:35.296408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84042b03-f11d-4828-91c3-661277e1e510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:35.297972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84042b03-f11d-4828-91c3-661277e1e510 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:35.304070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 248/491] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:35.305045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1071a-3e52-4775-9048-a7cb693450f0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.312134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 244/492] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:35.313898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84042b03-f11d-4828-91c3-661277e1e510 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.327795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:35.366158 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:35.413356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1071a-3e52-4775-9048-a7cb693450f0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.422503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84042b03-f11d-4828-91c3-661277e1e510 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.450238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c50c7-2af2-4229-8066-2a5d04ef269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.451451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c50c7-2af2-4229-8066-2a5d04ef269a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.451924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c50c7-2af2-4229-8066-2a5d04ef269a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.452393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c50c7-2af2-4229-8066-2a5d04ef269a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.458842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92adc80-8233-49f0-9704-0be30ae16a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.459717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92adc80-8233-49f0-9704-0be30ae16a98 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.460009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92adc80-8233-49f0-9704-0be30ae16a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.460293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92adc80-8233-49f0-9704-0be30ae16a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.460539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c50c7-2af2-4229-8066-2a5d04ef269a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:35.462004 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c50c7-2af2-4229-8066-2a5d04ef269a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:35.471763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92adc80-8233-49f0-9704-0be30ae16a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:35.473332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92adc80-8233-49f0-9704-0be30ae16a98 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d22c50c7-2af2-4229-8066-2a5d04ef269a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:35.477684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:35.478535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:35.479407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 249/493] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:35.480645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c50c7-2af2-4229-8066-2a5d04ef269a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e92adc80-8233-49f0-9704-0be30ae16a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:35.491595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:35.492467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:35.493301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:35.493301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:35.493301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:35.493301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:35.493301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 245/494] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:35.494300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4241b-31ef-4f99-a991-c2c9533c9f34 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.494482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4241b-31ef-4f99-a991-c2c9533c9f34 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.494823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4241b-31ef-4f99-a991-c2c9533c9f34 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.496101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 246/495] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:35.496941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4241b-31ef-4f99-a991-c2c9533c9f34 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.568364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:35.596784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c50c7-2af2-4229-8066-2a5d04ef269a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.611098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4241b-31ef-4f99-a991-c2c9533c9f34 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.612294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:35.635071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c2643-4e3d-4e65-bb40-3e14896d04e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.635913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c2643-4e3d-4e65-bb40-3e14896d04e4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.636201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c2643-4e3d-4e65-bb40-3e14896d04e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.636476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c2643-4e3d-4e65-bb40-3e14896d04e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.647085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c2643-4e3d-4e65-bb40-3e14896d04e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:35.647618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8216d-9251-4c4b-8c10-dbf7249bfa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.648494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8216d-9251-4c4b-8c10-dbf7249bfa04 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.649010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8216d-9251-4c4b-8c10-dbf7249bfa04 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.649492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8216d-9251-4c4b-8c10-dbf7249bfa04 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.650524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8216d-9251-4c4b-8c10-dbf7249bfa04 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:35.651028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8216d-9251-4c4b-8c10-dbf7249bfa04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:35.652807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8216d-9251-4c4b-8c10-dbf7249bfa04 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-471c2643-4e3d-4e65-bb40-3e14896d04e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:35.654913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:35.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:35.656570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 250/496] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:35.657764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c2643-4e3d-4e65-bb40-3e14896d04e4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.692487 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 247/497] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:35.693792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577464d4-9e31-4bf8-96d6-c0a361099aa0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.694086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577464d4-9e31-4bf8-96d6-c0a361099aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.694371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577464d4-9e31-4bf8-96d6-c0a361099aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.711331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-577464d4-9e31-4bf8-96d6-c0a361099aa0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:35.712550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-577464d4-9e31-4bf8-96d6-c0a361099aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:35.763777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c2643-4e3d-4e65-bb40-3e14896d04e4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.779393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 248/498] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:35.780266 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577464d4-9e31-4bf8-96d6-c0a361099aa0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.807668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce3056-6ce4-4a84-8a65-c6afc384427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.808340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce3056-6ce4-4a84-8a65-c6afc384427d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.808626 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce3056-6ce4-4a84-8a65-c6afc384427d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.808968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce3056-6ce4-4a84-8a65-c6afc384427d None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.809698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce3056-6ce4-4a84-8a65-c6afc384427d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:35.810009 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce3056-6ce4-4a84-8a65-c6afc384427d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:35.811488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce3056-6ce4-4a84-8a65-c6afc384427d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:35.823314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 251/499] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:35.824473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce3056-6ce4-4a84-8a65-c6afc384427d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.828267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:35.862208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:35.877450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577464d4-9e31-4bf8-96d6-c0a361099aa0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.910202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c381dae-a0b5-4837-a042-feb8d55b5a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.910858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c381dae-a0b5-4837-a042-feb8d55b5a67 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.911144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c381dae-a0b5-4837-a042-feb8d55b5a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.911425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c381dae-a0b5-4837-a042-feb8d55b5a67 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.912206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c381dae-a0b5-4837-a042-feb8d55b5a67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:35.912516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c381dae-a0b5-4837-a042-feb8d55b5a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:35.913907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c381dae-a0b5-4837-a042-feb8d55b5a67 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:35.921955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 249/500] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:35.923165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1c14d-376a-4df5-9710-4479d22ee3fe None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.923451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1c14d-376a-4df5-9710-4479d22ee3fe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.923753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1c14d-376a-4df5-9710-4479d22ee3fe None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:35.925154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 250/501] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:35.925646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1c14d-376a-4df5-9710-4479d22ee3fe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:35.933334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce3056-6ce4-4a84-8a65-c6afc384427d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:35.968835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c1423-f025-49af-a6ff-a46df6cf6616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:35.969481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c1423-f025-49af-a6ff-a46df6cf6616 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:35.969761 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c1423-f025-49af-a6ff-a46df6cf6616 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:35.970037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c1423-f025-49af-a6ff-a46df6cf6616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.025890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1c14d-376a-4df5-9710-4479d22ee3fe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.038486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:36.039036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c1423-f025-49af-a6ff-a46df6cf6616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.040602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c1423-f025-49af-a6ff-a46df6cf6616 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.059367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0899339-535f-425a-a536-13632d66f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.060193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0899339-535f-425a-a536-13632d66f154 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.060494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0899339-535f-425a-a536-13632d66f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.060756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0899339-535f-425a-a536-13632d66f154 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.071221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0899339-535f-425a-a536-13632d66f154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:36.075927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f0899339-535f-425a-a536-13632d66f154 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:36.079614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:36.080556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:36.081335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 251/502] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:36.082192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0899339-535f-425a-a536-13632d66f154 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.141783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 252/503] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:36.142731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c1423-f025-49af-a6ff-a46df6cf6616 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.180582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0899339-535f-425a-a536-13632d66f154 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.216507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088ab8-492e-4c65-9fe8-502564c63c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.217318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088ab8-492e-4c65-9fe8-502564c63c4d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.217639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088ab8-492e-4c65-9fe8-502564c63c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.217906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088ab8-492e-4c65-9fe8-502564c63c4d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.225504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4088ab8-492e-4c65-9fe8-502564c63c4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.226970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4088ab8-492e-4c65-9fe8-502564c63c4d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.240093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c1423-f025-49af-a6ff-a46df6cf6616 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e4088ab8-492e-4c65-9fe8-502564c63c4d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:36.242230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:36.243109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:36.244017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 252/504] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:36.244893 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd174e-2c7c-482b-a370-5c44c2cd6ba1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.244893 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd174e-2c7c-482b-a370-5c44c2cd6ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.245241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd174e-2c7c-482b-a370-5c44c2cd6ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.260910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-50dd174e-2c7c-482b-a370-5c44c2cd6ba1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:36.262178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-50dd174e-2c7c-482b-a370-5c44c2cd6ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:36.275371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f5b8-dcaa-4ff9-9d08-15da0d1b559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.276261 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f5b8-dcaa-4ff9-9d08-15da0d1b559b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.276492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f5b8-dcaa-4ff9-9d08-15da0d1b559b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.276775 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f5b8-dcaa-4ff9-9d08-15da0d1b559b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.278699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:36.287839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111f5b8-dcaa-4ff9-9d08-15da0d1b559b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.287995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:36.289294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111f5b8-dcaa-4ff9-9d08-15da0d1b559b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e111f5b8-dcaa-4ff9-9d08-15da0d1b559b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:36.306177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:36.307079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:36.307941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:36.307941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:36.307941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:36.307941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:36.307941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 253/505] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:36.308202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f5b8-dcaa-4ff9-9d08-15da0d1b559b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.330933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 253/506] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:36.331768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd174e-2c7c-482b-a370-5c44c2cd6ba1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.419537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f5b8-dcaa-4ff9-9d08-15da0d1b559b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.429610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd174e-2c7c-482b-a370-5c44c2cd6ba1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.451966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2843ae-778a-47c3-9907-43441bfc7695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.452672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2843ae-778a-47c3-9907-43441bfc7695 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.452960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2843ae-778a-47c3-9907-43441bfc7695 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.453240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2843ae-778a-47c3-9907-43441bfc7695 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.453989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2843ae-778a-47c3-9907-43441bfc7695 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:36.454288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2843ae-778a-47c3-9907-43441bfc7695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.455728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2843ae-778a-47c3-9907-43441bfc7695 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.462930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3edb5-3723-4613-be77-18b706317c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.463561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 254/507] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:36.463737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3edb5-3723-4613-be77-18b706317c15 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.464021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3edb5-3723-4613-be77-18b706317c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.464296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3edb5-3723-4613-be77-18b706317c15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.464387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2843ae-778a-47c3-9907-43441bfc7695 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.476818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3edb5-3723-4613-be77-18b706317c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.478307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3edb5-3723-4613-be77-18b706317c15 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c0c3edb5-3723-4613-be77-18b706317c15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:36.496395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:36.497221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:36.498394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:36.498394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:36.498394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:36.498394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:36.498394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 254/508] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:36.498394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3edb5-3723-4613-be77-18b706317c15 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.513378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:36.513832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:36.571796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2843ae-778a-47c3-9907-43441bfc7695 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.601985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3edb5-3723-4613-be77-18b706317c15 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.605069 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f075d-0aed-47cd-9575-205a037b1757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.605749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f075d-0aed-47cd-9575-205a037b1757 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.606041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f075d-0aed-47cd-9575-205a037b1757 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.606323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f075d-0aed-47cd-9575-205a037b1757 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.607163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f075d-0aed-47cd-9575-205a037b1757 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:36.607501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f075d-0aed-47cd-9575-205a037b1757 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.608975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f075d-0aed-47cd-9575-205a037b1757 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.634465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a9837-b41c-4037-8f17-832356406001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.635116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a9837-b41c-4037-8f17-832356406001 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.635376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a9837-b41c-4037-8f17-832356406001 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.635696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a9837-b41c-4037-8f17-832356406001 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.636407 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a9837-b41c-4037-8f17-832356406001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:36.636710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a9837-b41c-4037-8f17-832356406001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.638099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a9837-b41c-4037-8f17-832356406001 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.645101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 255/509] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:36.645921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f075d-0aed-47cd-9575-205a037b1757 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.650035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 255/510] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:36.650717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a9837-b41c-4037-8f17-832356406001 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.725682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:36.747914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f075d-0aed-47cd-9575-205a037b1757 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.751971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a9837-b41c-4037-8f17-832356406001 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.755234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:36.781706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eb6a9-53e3-4268-a76f-8c618b703ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.782536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eb6a9-53e3-4268-a76f-8c618b703ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.782829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eb6a9-53e3-4268-a76f-8c618b703ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.783086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eb6a9-53e3-4268-a76f-8c618b703ab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.786577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76436b-d5e0-4264-9676-458d6579c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.787377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76436b-d5e0-4264-9676-458d6579c352 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.787681 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76436b-d5e0-4264-9676-458d6579c352 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.787945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76436b-d5e0-4264-9676-458d6579c352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.793847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223eb6a9-53e3-4268-a76f-8c618b703ab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.795332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223eb6a9-53e3-4268-a76f-8c618b703ab5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.798553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76436b-d5e0-4264-9676-458d6579c352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.800041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76436b-d5e0-4264-9676-458d6579c352 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-223eb6a9-53e3-4268-a76f-8c618b703ab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:36.812767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:36.813589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:36.814415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:36.814415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:36.814415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:36.814415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:36.814415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 256/511] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:36.814676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eb6a9-53e3-4268-a76f-8c618b703ab5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-dc76436b-d5e0-4264-9676-458d6579c352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:36.817672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:36.818682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:36.819527 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:36.819527 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:36.819527 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:36.819527 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:36.819527 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 256/512] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:36.820308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76436b-d5e0-4264-9676-458d6579c352 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.919380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eb6a9-53e3-4268-a76f-8c618b703ab5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.921141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76436b-d5e0-4264-9676-458d6579c352 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:36.947112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:36.954449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b26b7-3c63-4056-a583-6309c14edad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.955199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b26b7-3c63-4056-a583-6309c14edad9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.955583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b26b7-3c63-4056-a583-6309c14edad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.955859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b26b7-3c63-4056-a583-6309c14edad9 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.956539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966084b4-ddc6-450a-9d16-c80091e72439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:36.956667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b26b7-3c63-4056-a583-6309c14edad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:36.956880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b26b7-3c63-4056-a583-6309c14edad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:36.957198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966084b4-ddc6-450a-9d16-c80091e72439 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:36.957484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966084b4-ddc6-450a-9d16-c80091e72439 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:36.957764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966084b4-ddc6-450a-9d16-c80091e72439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:36.958289 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b26b7-3c63-4056-a583-6309c14edad9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:36.970002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 257/513] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:36.974509 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b26b7-3c63-4056-a583-6309c14edad9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:36.992220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:37.023787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966084b4-ddc6-450a-9d16-c80091e72439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.025235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966084b4-ddc6-450a-9d16-c80091e72439 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.075077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b26b7-3c63-4056-a583-6309c14edad9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.109104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200656-becc-45f2-afd6-277d0e698527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.109910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200656-becc-45f2-afd6-277d0e698527 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.110202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200656-becc-45f2-afd6-277d0e698527 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.110481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200656-becc-45f2-afd6-277d0e698527 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.117472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200656-becc-45f2-afd6-277d0e698527 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.118944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200656-becc-45f2-afd6-277d0e698527 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.127560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 257/514] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:37.128392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966084b4-ddc6-450a-9d16-c80091e72439 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b4200656-becc-45f2-afd6-277d0e698527 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:37.133681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:37.134697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:37.135597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 258/515] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:37.136144 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200656-becc-45f2-afd6-277d0e698527 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.154595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:37.210430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:37.230170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966084b4-ddc6-450a-9d16-c80091e72439 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.231989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200656-becc-45f2-afd6-277d0e698527 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.269017 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b3fcd-c484-42ae-a447-601c8d363558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.269683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b3fcd-c484-42ae-a447-601c8d363558 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.269965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b3fcd-c484-42ae-a447-601c8d363558 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.270239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b3fcd-c484-42ae-a447-601c8d363558 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.270962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b3fcd-c484-42ae-a447-601c8d363558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:37.271268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b3fcd-c484-42ae-a447-601c8d363558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.272772 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b3fcd-c484-42ae-a447-601c8d363558 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.273044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cf0ee-2f47-4e09-a265-0ff921edf5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.273712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cf0ee-2f47-4e09-a265-0ff921edf5bb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.274003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cf0ee-2f47-4e09-a265-0ff921edf5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.274281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cf0ee-2f47-4e09-a265-0ff921edf5bb None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.275124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cf0ee-2f47-4e09-a265-0ff921edf5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:37.275572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cf0ee-2f47-4e09-a265-0ff921edf5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.276941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cf0ee-2f47-4e09-a265-0ff921edf5bb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.284097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 259/516] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:37.284903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b3fcd-c484-42ae-a447-601c8d363558 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.293834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 258/517] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:37.294643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cf0ee-2f47-4e09-a265-0ff921edf5bb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.377721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:37.387018 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b3fcd-c484-42ae-a447-601c8d363558 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.392989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cf0ee-2f47-4e09-a265-0ff921edf5bb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.420025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d55b5-3e26-446b-9cbf-385ecccae1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.420821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d55b5-3e26-446b-9cbf-385ecccae1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.421108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d55b5-3e26-446b-9cbf-385ecccae1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.421384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d55b5-3e26-446b-9cbf-385ecccae1e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.424975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fc43d-11ec-40f9-9bf8-ee347af6b6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.425777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fc43d-11ec-40f9-9bf8-ee347af6b6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.426067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fc43d-11ec-40f9-9bf8-ee347af6b6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.426348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fc43d-11ec-40f9-9bf8-ee347af6b6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.427920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d55b5-3e26-446b-9cbf-385ecccae1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.429380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d55b5-3e26-446b-9cbf-385ecccae1e8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.431073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:37.437090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fc43d-11ec-40f9-9bf8-ee347af6b6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.438543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fc43d-11ec-40f9-9bf8-ee347af6b6f8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b12d55b5-3e26-446b-9cbf-385ecccae1e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:37.445185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:37.445982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:37.446807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:37.446807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:37.446807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:37.446807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:37.446807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:37.446807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:37.446807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:37.446807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:37.446807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 260/518] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:37.447261 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d55b5-3e26-446b-9cbf-385ecccae1e8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-129fc43d-11ec-40f9-9bf8-ee347af6b6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:37.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:37.463395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:37.464286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:37.464286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:37.464286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:37.464286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:37.464286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 259/519] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:37.464286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fc43d-11ec-40f9-9bf8-ee347af6b6f8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.549673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d55b5-3e26-446b-9cbf-385ecccae1e8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.563155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fc43d-11ec-40f9-9bf8-ee347af6b6f8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.585647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df690c-3cd5-45ab-9f49-f9d55b25b574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.586323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df690c-3cd5-45ab-9f49-f9d55b25b574 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.586621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df690c-3cd5-45ab-9f49-f9d55b25b574 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.586891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df690c-3cd5-45ab-9f49-f9d55b25b574 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.587770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df690c-3cd5-45ab-9f49-f9d55b25b574 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:37.588070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df690c-3cd5-45ab-9f49-f9d55b25b574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.589533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df690c-3cd5-45ab-9f49-f9d55b25b574 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.597309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1185fc-5b46-412f-ad6d-c5ad82642d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.598105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1185fc-5b46-412f-ad6d-c5ad82642d8b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.598389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1185fc-5b46-412f-ad6d-c5ad82642d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.598664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1185fc-5b46-412f-ad6d-c5ad82642d8b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.606020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1185fc-5b46-412f-ad6d-c5ad82642d8b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.606194 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 261/520] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:37.606924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df690c-3cd5-45ab-9f49-f9d55b25b574 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.607482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1185fc-5b46-412f-ad6d-c5ad82642d8b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.609204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1d1185fc-5b46-412f-ad6d-c5ad82642d8b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:37.624248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:37.625162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:37.626040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 260/521] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:37.626582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1185fc-5b46-412f-ad6d-c5ad82642d8b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.665739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:37.707800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df690c-3cd5-45ab-9f49-f9d55b25b574 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.725680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1185fc-5b46-412f-ad6d-c5ad82642d8b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.740744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6cfe-1a11-4011-90d3-b15772c8fc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.741416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6cfe-1a11-4011-90d3-b15772c8fc67 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.741712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6cfe-1a11-4011-90d3-b15772c8fc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.741986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6cfe-1a11-4011-90d3-b15772c8fc67 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.742687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac6cfe-1a11-4011-90d3-b15772c8fc67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.744190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac6cfe-1a11-4011-90d3-b15772c8fc67 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.759531 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d357ad2-fd4d-4765-902e-2ed7ebce4d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.760296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d357ad2-fd4d-4765-902e-2ed7ebce4d3b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.760577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d357ad2-fd4d-4765-902e-2ed7ebce4d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.760851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d357ad2-fd4d-4765-902e-2ed7ebce4d3b None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.761624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d357ad2-fd4d-4765-902e-2ed7ebce4d3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:37.762056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d357ad2-fd4d-4765-902e-2ed7ebce4d3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.763492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d357ad2-fd4d-4765-902e-2ed7ebce4d3b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.764142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 262/522] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:28:37 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:28:37.764940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6cfe-1a11-4011-90d3-b15772c8fc67 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.787760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 261/523] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:37.787760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d357ad2-fd4d-4765-902e-2ed7ebce4d3b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:37.849201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:37.882968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6cfe-1a11-4011-90d3-b15772c8fc67 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.895897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d357ad2-fd4d-4765-902e-2ed7ebce4d3b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:37.898238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:37.924059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f65e60-4a7d-4545-b956-caed7074d389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.924742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f65e60-4a7d-4545-b956-caed7074d389 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.925031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f65e60-4a7d-4545-b956-caed7074d389 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.925315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f65e60-4a7d-4545-b956-caed7074d389 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.926049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f65e60-4a7d-4545-b956-caed7074d389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:37.926361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f65e60-4a7d-4545-b956-caed7074d389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:37.927862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f65e60-4a7d-4545-b956-caed7074d389 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:37.939069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba8fe6-f056-4d92-b2c7-894cdd5b3be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:37.939761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba8fe6-f056-4d92-b2c7-894cdd5b3be7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:37.940037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba8fe6-f056-4d92-b2c7-894cdd5b3be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:37.943499 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba8fe6-f056-4d92-b2c7-894cdd5b3be7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:37.943620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 263/524] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:37.943620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f65e60-4a7d-4545-b956-caed7074d389 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.015759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba8fe6-f056-4d92-b2c7-894cdd5b3be7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:38.017151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba8fe6-f056-4d92-b2c7-894cdd5b3be7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:38.058508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f65e60-4a7d-4545-b956-caed7074d389 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:38.078657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:38.097603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3ffb-7c43-4e91-ad3a-6bd04e8e3060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:38.098372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3ffb-7c43-4e91-ad3a-6bd04e8e3060 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:38.098859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3ffb-7c43-4e91-ad3a-6bd04e8e3060 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:38.099090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3ffb-7c43-4e91-ad3a-6bd04e8e3060 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:38.100172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3ffb-7c43-4e91-ad3a-6bd04e8e3060 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:38.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3ffb-7c43-4e91-ad3a-6bd04e8e3060 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:38.101954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3ffb-7c43-4e91-ad3a-6bd04e8e3060 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:38.125236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 264/525] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:38.125236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3ffb-7c43-4e91-ad3a-6bd04e8e3060 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.126641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 262/526] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:38.127556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba8fe6-f056-4d92-b2c7-894cdd5b3be7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.160207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:38.234612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba8fe6-f056-4d92-b2c7-894cdd5b3be7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:38.235148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3ffb-7c43-4e91-ad3a-6bd04e8e3060 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:38.269539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64bfa1-5f5c-48ac-84c2-033b7393ae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:38.270130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64bfa1-5f5c-48ac-84c2-033b7393ae3a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:38.270403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64bfa1-5f5c-48ac-84c2-033b7393ae3a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:38.270684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64bfa1-5f5c-48ac-84c2-033b7393ae3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:38.271010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875e7c9-84b9-4e29-8e23-799ad7394beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:38.271539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64bfa1-5f5c-48ac-84c2-033b7393ae3a None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=e80bb5b10ce945cabbfb88ca5df92e3c) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:38.271849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875e7c9-84b9-4e29-8e23-799ad7394beb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:38.271945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64bfa1-5f5c-48ac-84c2-033b7393ae3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:38.272132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875e7c9-84b9-4e29-8e23-799ad7394beb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:38.272403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875e7c9-84b9-4e29-8e23-799ad7394beb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:38.273294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64bfa1-5f5c-48ac-84c2-033b7393ae3a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:38.286140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 265/527] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:38 2022] GET /identity/v3/role_assignments?user.id=e80bb5b10ce945cabbfb88ca5df92e3c&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:38.286335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875e7c9-84b9-4e29-8e23-799ad7394beb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:38.286427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64bfa1-5f5c-48ac-84c2-033b7393ae3a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.286541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875e7c9-84b9-4e29-8e23-799ad7394beb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a875e7c9-84b9-4e29-8e23-799ad7394beb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:38.306498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:38.307416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:38.309006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:38.309006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:38.309006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:38.309006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:38.309006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:38.309006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:38.309006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:38.309006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:38.309006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 263/528] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:38.309462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875e7c9-84b9-4e29-8e23-799ad7394beb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.330389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:38.403233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:38.413361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64bfa1-5f5c-48ac-84c2-033b7393ae3a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:38.422261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875e7c9-84b9-4e29-8e23-799ad7394beb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:38.459763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220096b2-aed5-4886-9218-1e6ca07bdd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:38.460396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67e341-35e0-49dc-ada6-488b6fa256d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:38.460515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220096b2-aed5-4886-9218-1e6ca07bdd29 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:38.460697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220096b2-aed5-4886-9218-1e6ca07bdd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:38.460970 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220096b2-aed5-4886-9218-1e6ca07bdd29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:38.461924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67e341-35e0-49dc-ada6-488b6fa256d5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:38.462404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67e341-35e0-49dc-ada6-488b6fa256d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:38.462872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67e341-35e0-49dc-ada6-488b6fa256d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:38.474967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67e341-35e0-49dc-ada6-488b6fa256d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:38.476841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67e341-35e0-49dc-ada6-488b6fa256d5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-aa67e341-35e0-49dc-ada6-488b6fa256d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:38.497378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:38.498293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:38.499118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:38.499118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:38.499118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:38.499118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:38.501085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 264/529] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:38.502156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67e341-35e0-49dc-ada6-488b6fa256d5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.547995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220096b2-aed5-4886-9218-1e6ca07bdd29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:38.549556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220096b2-aed5-4886-9218-1e6ca07bdd29 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:38.580653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:38.606492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67e341-35e0-49dc-ada6-488b6fa256d5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:38.640063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7329fdb-12ee-44d4-82b4-da32dac8ddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:38.640730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7329fdb-12ee-44d4-82b4-da32dac8ddee None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:38.641016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7329fdb-12ee-44d4-82b4-da32dac8ddee None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:38.641295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7329fdb-12ee-44d4-82b4-da32dac8ddee None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:38.642084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7329fdb-12ee-44d4-82b4-da32dac8ddee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:38.642395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7329fdb-12ee-44d4-82b4-da32dac8ddee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:38.643879 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7329fdb-12ee-44d4-82b4-da32dac8ddee None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:38.659496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 265/530] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:38.659496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7329fdb-12ee-44d4-82b4-da32dac8ddee None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.663963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:38.673638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 266/531] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:38.674873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220096b2-aed5-4886-9218-1e6ca07bdd29 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.763962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7329fdb-12ee-44d4-82b4-da32dac8ddee None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:38.785486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220096b2-aed5-4886-9218-1e6ca07bdd29 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:38.793041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:38.800198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee8df0-d87d-44fa-a88c-005d415dcc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:38.800885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee8df0-d87d-44fa-a88c-005d415dcc22 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:38.801236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee8df0-d87d-44fa-a88c-005d415dcc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:38.801580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee8df0-d87d-44fa-a88c-005d415dcc22 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:38.802377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee8df0-d87d-44fa-a88c-005d415dcc22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:38.802770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee8df0-d87d-44fa-a88c-005d415dcc22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:38.804253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee8df0-d87d-44fa-a88c-005d415dcc22 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:38.816482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 266/532] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:38.817199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee8df0-d87d-44fa-a88c-005d415dcc22 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.822950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0769d-40e8-4adb-b3f6-1531a1bdd839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:38.823966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0769d-40e8-4adb-b3f6-1531a1bdd839 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:38.824290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0769d-40e8-4adb-b3f6-1531a1bdd839 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:38.824573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0769d-40e8-4adb-b3f6-1531a1bdd839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:38.825420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0769d-40e8-4adb-b3f6-1531a1bdd839 None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=cc21faf2237d405bba03e5309d93629f) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:38.825735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0769d-40e8-4adb-b3f6-1531a1bdd839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:38.827221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0769d-40e8-4adb-b3f6-1531a1bdd839 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:38.833348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 267/533] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:38 2022] GET /identity/v3/role_assignments?user.id=cc21faf2237d405bba03e5309d93629f&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:38.834325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0769d-40e8-4adb-b3f6-1531a1bdd839 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:38.901557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:38.957931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0769d-40e8-4adb-b3f6-1531a1bdd839 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:39.007942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019edc24-a23b-4c0f-b269-22e0b1209ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:39.009140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019edc24-a23b-4c0f-b269-22e0b1209ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:39.009659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019edc24-a23b-4c0f-b269-22e0b1209ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:39.010177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019edc24-a23b-4c0f-b269-22e0b1209ee9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:39.015600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee8df0-d87d-44fa-a88c-005d415dcc22 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:39.019601 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019edc24-a23b-4c0f-b269-22e0b1209ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:39.021400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019edc24-a23b-4c0f-b269-22e0b1209ee9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-019edc24-a23b-4c0f-b269-22e0b1209ee9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:39.039317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:39.040227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:39.041041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:39.041041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:39.041041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:39.041041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:39.041041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:39.041041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:39.041041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:39.041041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:39.047785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 268/534] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:39.050287 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019edc24-a23b-4c0f-b269-22e0b1209ee9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:39.094589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec16dea-8322-457b-9016-10aef6883cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:39.095406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec16dea-8322-457b-9016-10aef6883cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:39.095806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec16dea-8322-457b-9016-10aef6883cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:39.096191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec16dea-8322-457b-9016-10aef6883cc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:39.120892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec16dea-8322-457b-9016-10aef6883cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:39.122620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec16dea-8322-457b-9016-10aef6883cc1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0ec16dea-8322-457b-9016-10aef6883cc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:39.146334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:39.147374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:39.148446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:39.148446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:39.148446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:39.148446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:39.159011 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 267/535] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:39.165746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec16dea-8322-457b-9016-10aef6883cc1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:39.205538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019edc24-a23b-4c0f-b269-22e0b1209ee9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:39.234898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:39.239801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:39.269087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743692a-7d8d-4850-aeed-f98f5c7f853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:39.270347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743692a-7d8d-4850-aeed-f98f5c7f853d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:39.272073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743692a-7d8d-4850-aeed-f98f5c7f853d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:39.274735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743692a-7d8d-4850-aeed-f98f5c7f853d None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:39.275824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743692a-7d8d-4850-aeed-f98f5c7f853d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:39.276339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743692a-7d8d-4850-aeed-f98f5c7f853d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:39.278285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743692a-7d8d-4850-aeed-f98f5c7f853d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:39.314732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 269/536] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:39.322460 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743692a-7d8d-4850-aeed-f98f5c7f853d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:39.324479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec16dea-8322-457b-9016-10aef6883cc1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:39.399715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83227e50-f034-41b6-86fe-5e91dc22882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:39.400800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83227e50-f034-41b6-86fe-5e91dc22882b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:39.401284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83227e50-f034-41b6-86fe-5e91dc22882b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:39.401774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83227e50-f034-41b6-86fe-5e91dc22882b None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:39.402835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83227e50-f034-41b6-86fe-5e91dc22882b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:39.405703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83227e50-f034-41b6-86fe-5e91dc22882b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:39.407138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83227e50-f034-41b6-86fe-5e91dc22882b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:39.422112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 268/537] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:39.427811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83227e50-f034-41b6-86fe-5e91dc22882b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:39.488829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743692a-7d8d-4850-aeed-f98f5c7f853d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:39.554498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bb968-0fcd-45da-9c01-27eb78f16ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:39.555567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bb968-0fcd-45da-9c01-27eb78f16ffb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:39.556087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bb968-0fcd-45da-9c01-27eb78f16ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:39.556562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bb968-0fcd-45da-9c01-27eb78f16ffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:39.557607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bb968-0fcd-45da-9c01-27eb78f16ffb None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=9fc23c141eff42f98d31deb2d8aaaca6) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:39.559525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bb968-0fcd-45da-9c01-27eb78f16ffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:39.561307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bb968-0fcd-45da-9c01-27eb78f16ffb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:39.571328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 270/538] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:39 2022] GET /identity/v3/role_assignments?user.id=9fc23c141eff42f98d31deb2d8aaaca6&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:39.584177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83227e50-f034-41b6-86fe-5e91dc22882b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:39.588018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:39.633664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7496ec7-4241-4775-881b-a04420256dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:39.634886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7496ec7-4241-4775-881b-a04420256dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:39.635558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7496ec7-4241-4775-881b-a04420256dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:39.636035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7496ec7-4241-4775-881b-a04420256dd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:39.643910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7496ec7-4241-4775-881b-a04420256dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:39.645688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7496ec7-4241-4775-881b-a04420256dd2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a7496ec7-4241-4775-881b-a04420256dd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:39.661146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:39.662029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:39.662866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:39.662866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:39.662866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:39.662866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:39.662866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:39.662866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:39.662866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:39.662866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:39.665443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 269/539] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:39.676706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bb968-0fcd-45da-9c01-27eb78f16ffb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:39.752078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:39.834279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bb968-0fcd-45da-9c01-27eb78f16ffb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:39.883480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccbf02-38ad-418b-9fd6-2abcba589395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:39.884551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccbf02-38ad-418b-9fd6-2abcba589395 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:39.885058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccbf02-38ad-418b-9fd6-2abcba589395 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:39.885660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccbf02-38ad-418b-9fd6-2abcba589395 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:39.886646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccbf02-38ad-418b-9fd6-2abcba589395 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:39.887258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccbf02-38ad-418b-9fd6-2abcba589395 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:39.888964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccbf02-38ad-418b-9fd6-2abcba589395 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:39.902759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 271/540] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:39.908318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7496ec7-4241-4775-881b-a04420256dd2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:39.943921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bccb16-2e1e-4fda-971f-3e649a9faf1c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:39.944530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bccb16-2e1e-4fda-971f-3e649a9faf1c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:39.945049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bccb16-2e1e-4fda-971f-3e649a9faf1c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:39.947071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 272/541] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:39.956685 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62379-59df-445f-a641-22d198cac4e1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:39.957237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62379-59df-445f-a641-22d198cac4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:39.957737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62379-59df-445f-a641-22d198cac4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:39.988478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6ff62379-59df-445f-a641-22d198cac4e1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:39.990077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6ff62379-59df-445f-a641-22d198cac4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:40.084882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 273/542] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:40.093887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7496ec7-4241-4775-881b-a04420256dd2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:40.120290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62379-59df-445f-a641-22d198cac4e1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:40.138076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed8392-6f96-4b4c-9f26-e4082ad33ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:40.138747 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed8392-6f96-4b4c-9f26-e4082ad33ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:40.139033 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed8392-6f96-4b4c-9f26-e4082ad33ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:40.139314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed8392-6f96-4b4c-9f26-e4082ad33ec8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:40.140177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed8392-6f96-4b4c-9f26-e4082ad33ec8 None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=c57d44c19d15496e96c6e37510ce67fc) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:40.140492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed8392-6f96-4b4c-9f26-e4082ad33ec8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:40.141944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed8392-6f96-4b4c-9f26-e4082ad33ec8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:40.148206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 270/543] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:39 2022] GET /identity/v3/role_assignments?user.id=c57d44c19d15496e96c6e37510ce67fc&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:40.160390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:40.310378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62379-59df-445f-a641-22d198cac4e1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:40.352554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45611ce6-83e0-403e-9c8f-baf25c720a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:40.353288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45611ce6-83e0-403e-9c8f-baf25c720a7a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:40.353573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45611ce6-83e0-403e-9c8f-baf25c720a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:40.353850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45611ce6-83e0-403e-9c8f-baf25c720a7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:40.372313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45611ce6-83e0-403e-9c8f-baf25c720a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-45611ce6-83e0-403e-9c8f-baf25c720a7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:40.380054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:40.380890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:40.381657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 274/544] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:40.387752 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed8392-6f96-4b4c-9f26-e4082ad33ec8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:40.459845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:40.562364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed8392-6f96-4b4c-9f26-e4082ad33ec8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:40.607754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbdb2b-d48f-4190-84a3-6c8cb20d2405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:40.608789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbdb2b-d48f-4190-84a3-6c8cb20d2405 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:40.609270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbdb2b-d48f-4190-84a3-6c8cb20d2405 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:40.609749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbdb2b-d48f-4190-84a3-6c8cb20d2405 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:40.610759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbdb2b-d48f-4190-84a3-6c8cb20d2405 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:40.611263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbdb2b-d48f-4190-84a3-6c8cb20d2405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:40.613026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbdb2b-d48f-4190-84a3-6c8cb20d2405 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:40.626709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 271/545] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:40.628370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45611ce6-83e0-403e-9c8f-baf25c720a7a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:40.758978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:40.768113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45611ce6-83e0-403e-9c8f-baf25c720a7a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:40.815010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab4e66-82a5-4eb6-ba57-dd1a0f3e4a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:40.816190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab4e66-82a5-4eb6-ba57-dd1a0f3e4a95 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:40.816679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab4e66-82a5-4eb6-ba57-dd1a0f3e4a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:40.819007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ca83d-657d-4e81-b700-50f62fd0dea9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:40.819565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ca83d-657d-4e81-b700-50f62fd0dea9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:40.820096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ca83d-657d-4e81-b700-50f62fd0dea9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:40.822047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 272/546] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:40.822318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab4e66-82a5-4eb6-ba57-dd1a0f3e4a95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:40.827230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457df0a3-a47e-420f-8c6d-91c9c4021db7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:40.827551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457df0a3-a47e-420f-8c6d-91c9c4021db7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:40.827837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457df0a3-a47e-420f-8c6d-91c9c4021db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:40.836669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab4e66-82a5-4eb6-ba57-dd1a0f3e4a95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:40.838531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab4e66-82a5-4eb6-ba57-dd1a0f3e4a95 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-36ab4e66-82a5-4eb6-ba57-dd1a0f3e4a95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:40.862001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:40.862862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:40.863735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:40.863735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:40.863735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:40.863735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:40.869812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-457df0a3-a47e-420f-8c6d-91c9c4021db7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:40.871302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 275/547] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:40.872881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-457df0a3-a47e-420f-8c6d-91c9c4021db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:40.876911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab4e66-82a5-4eb6-ba57-dd1a0f3e4a95 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:41.008417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 273/548] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:41.054931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457df0a3-a47e-420f-8c6d-91c9c4021db7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:41.086549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab4e66-82a5-4eb6-ba57-dd1a0f3e4a95 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:41.087290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:41.180300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457df0a3-a47e-420f-8c6d-91c9c4021db7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:41.193298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6a896-3b5c-496f-9093-30e0aa7d37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:41.193912 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a896-3b5c-496f-9093-30e0aa7d37bb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.194202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a896-3b5c-496f-9093-30e0aa7d37bb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.194481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a896-3b5c-496f-9093-30e0aa7d37bb None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.195217 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6a896-3b5c-496f-9093-30e0aa7d37bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:41.195552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6a896-3b5c-496f-9093-30e0aa7d37bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:41.197120 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6a896-3b5c-496f-9093-30e0aa7d37bb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:41.221692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 276/549] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:41.222538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b96318-8c3f-439f-991e-74f05a4a03ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:41.223535 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96318-8c3f-439f-991e-74f05a4a03ac None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.223955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96318-8c3f-439f-991e-74f05a4a03ac None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.224388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96318-8c3f-439f-991e-74f05a4a03ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.226591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6a896-3b5c-496f-9093-30e0aa7d37bb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:41.231938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:41.250616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b96318-8c3f-439f-991e-74f05a4a03ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-80b96318-8c3f-439f-991e-74f05a4a03ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:41.265791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:41.267430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:41.268602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:41.275745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 274/550] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:41.280164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b96318-8c3f-439f-991e-74f05a4a03ac None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:41.367321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6a896-3b5c-496f-9093-30e0aa7d37bb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:41.380684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:41.409475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b96318-8c3f-439f-991e-74f05a4a03ac None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:41.410084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793b7db-28f7-4210-87f3-e6c79916a1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:41.410854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b7db-28f7-4210-87f3-e6c79916a1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.411287 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b7db-28f7-4210-87f3-e6c79916a1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.411656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b7db-28f7-4210-87f3-e6c79916a1c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.422620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b7db-28f7-4210-87f3-e6c79916a1c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:41.424758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b7db-28f7-4210-87f3-e6c79916a1c2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5793b7db-28f7-4210-87f3-e6c79916a1c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:41.444269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:41.445326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:28:41.446958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:41.450916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 277/551] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:41.452383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b16f62-0a00-4e19-ae98-9580dd355600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:41.452533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4504b9-7d70-4921-9bb6-79e6f5cf104d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.452683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4504b9-7d70-4921-9bb6-79e6f5cf104d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.452966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4504b9-7d70-4921-9bb6-79e6f5cf104d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.453417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b16f62-0a00-4e19-ae98-9580dd355600 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.453800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b16f62-0a00-4e19-ae98-9580dd355600 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.454172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b16f62-0a00-4e19-ae98-9580dd355600 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.454490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 278/552] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:41.455188 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4504b9-7d70-4921-9bb6-79e6f5cf104d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:41.459047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b16f62-0a00-4e19-ae98-9580dd355600 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:41.459449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b16f62-0a00-4e19-ae98-9580dd355600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:41.461243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b16f62-0a00-4e19-ae98-9580dd355600 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:41.472943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 275/553] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:41.474366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d31917-2fa3-4a77-a910-7d696909b96f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.474713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d31917-2fa3-4a77-a910-7d696909b96f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.475052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d31917-2fa3-4a77-a910-7d696909b96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.497603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-09d31917-2fa3-4a77-a910-7d696909b96f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:41.499745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-09d31917-2fa3-4a77-a910-7d696909b96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:41.545115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:41.599567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 276/554] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:41.600412 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4504b9-7d70-4921-9bb6-79e6f5cf104d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:41.600575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d31917-2fa3-4a77-a910-7d696909b96f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:41.642870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31053fc9-de51-4f0c-982b-3ec72e137892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:41.643949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31053fc9-de51-4f0c-982b-3ec72e137892 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.644626 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31053fc9-de51-4f0c-982b-3ec72e137892 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.644914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31053fc9-de51-4f0c-982b-3ec72e137892 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.646020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31053fc9-de51-4f0c-982b-3ec72e137892 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:41.646336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31053fc9-de51-4f0c-982b-3ec72e137892 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:41.648087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31053fc9-de51-4f0c-982b-3ec72e137892 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:41.672038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 279/555] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 13 20:28:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:41.673039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31053fc9-de51-4f0c-982b-3ec72e137892 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:41.702970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:41.743965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d31917-2fa3-4a77-a910-7d696909b96f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:41.781159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31053fc9-de51-4f0c-982b-3ec72e137892 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:41.783345 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b6fe6-7c0e-4ca5-aa74-d2a319ff635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:41.784506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b6fe6-7c0e-4ca5-aa74-d2a319ff635b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.785002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b6fe6-7c0e-4ca5-aa74-d2a319ff635b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.785452 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b6fe6-7c0e-4ca5-aa74-d2a319ff635b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.797569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b6fe6-7c0e-4ca5-aa74-d2a319ff635b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:41.799554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b6fe6-7c0e-4ca5-aa74-d2a319ff635b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:41.805568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:41.817806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f031e8-27b1-4408-a637-a590405e94ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:41.818647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f031e8-27b1-4408-a637-a590405e94ce None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.818937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f031e8-27b1-4408-a637-a590405e94ce None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.819210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f031e8-27b1-4408-a637-a590405e94ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-173b6fe6-7c0e-4ca5-aa74-d2a319ff635b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:41.823117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:41.824342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:41.825181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:41.825181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:41.825181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:41.825181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:41.826930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 277/556] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:41.827964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b6fe6-7c0e-4ca5-aa74-d2a319ff635b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:41.835865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f031e8-27b1-4408-a637-a590405e94ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-89f031e8-27b1-4408-a637-a590405e94ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:41.844258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:41.845286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:41.846539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:41.850472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 280/557] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:41.851375 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f031e8-27b1-4408-a637-a590405e94ce None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:41.942621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b6fe6-7c0e-4ca5-aa74-d2a319ff635b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:41.955329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f031e8-27b1-4408-a637-a590405e94ce None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:41.969803 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:41.984144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22dc22-7f86-4426-ac00-1230c16b60a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:41.985275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22dc22-7f86-4426-ac00-1230c16b60a0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.985926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22dc22-7f86-4426-ac00-1230c16b60a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.986406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22dc22-7f86-4426-ac00-1230c16b60a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.990779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0e348-31f0-4405-bf51-36995959da20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:41.991494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0e348-31f0-4405-bf51-36995959da20 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:41.992007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0e348-31f0-4405-bf51-36995959da20 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:41.992521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0e348-31f0-4405-bf51-36995959da20 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:41.993468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0e348-31f0-4405-bf51-36995959da20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:41.993978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0e348-31f0-4405-bf51-36995959da20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:41.995620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0e348-31f0-4405-bf51-36995959da20 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.008489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 281/558] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:42.009802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0e348-31f0-4405-bf51-36995959da20 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.054496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:42.076470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22dc22-7f86-4426-ac00-1230c16b60a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:42.078146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22dc22-7f86-4426-ac00-1230c16b60a0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.115825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0e348-31f0-4405-bf51-36995959da20 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.155987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3f1d6-6c19-4e9c-8c02-c0bd03943ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:42.157151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3f1d6-6c19-4e9c-8c02-c0bd03943ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.157507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3f1d6-6c19-4e9c-8c02-c0bd03943ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.157792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3f1d6-6c19-4e9c-8c02-c0bd03943ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.158552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3f1d6-6c19-4e9c-8c02-c0bd03943ea4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:42.158981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3f1d6-6c19-4e9c-8c02-c0bd03943ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:42.160393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3f1d6-6c19-4e9c-8c02-c0bd03943ea4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.169050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 282/559] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:42.169690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3f1d6-6c19-4e9c-8c02-c0bd03943ea4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.186336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 278/560] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:42.187981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00809b1-4a8a-4056-8f40-a54293520287 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.188510 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00809b1-4a8a-4056-8f40-a54293520287 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.189016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00809b1-4a8a-4056-8f40-a54293520287 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.194491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 279/561] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:42.195505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00809b1-4a8a-4056-8f40-a54293520287 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.221618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:42.277567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3f1d6-6c19-4e9c-8c02-c0bd03943ea4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.280154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:42.299151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00809b1-4a8a-4056-8f40-a54293520287 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.312921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063280b1-c792-4dc9-b9a4-bece469e6f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:42.313759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063280b1-c792-4dc9-b9a4-bece469e6f2b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.314046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063280b1-c792-4dc9-b9a4-bece469e6f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.314321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063280b1-c792-4dc9-b9a4-bece469e6f2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.324139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063280b1-c792-4dc9-b9a4-bece469e6f2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:42.325625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063280b1-c792-4dc9-b9a4-bece469e6f2b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.333247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdd55c-e658-437d-8124-e95662e71905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:42.334064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdd55c-e658-437d-8124-e95662e71905 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.334356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdd55c-e658-437d-8124-e95662e71905 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.334632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdd55c-e658-437d-8124-e95662e71905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-063280b1-c792-4dc9-b9a4-bece469e6f2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:42.339327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:42.340401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:42.341311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 283/562] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:42.341974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063280b1-c792-4dc9-b9a4-bece469e6f2b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.349496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdd55c-e658-437d-8124-e95662e71905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:42.350932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdd55c-e658-437d-8124-e95662e71905 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c1bdd55c-e658-437d-8124-e95662e71905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:42.369224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:42.370286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:42.371259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:42.371259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:42.371259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:42.371259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:42.371259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 280/563] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:42.371917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b4fc4-fec1-4a35-bc0d-c3cbd47d1798 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.372073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b4fc4-fec1-4a35-bc0d-c3cbd47d1798 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.372489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b4fc4-fec1-4a35-bc0d-c3cbd47d1798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.389358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d67b4fc4-fec1-4a35-bc0d-c3cbd47d1798 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:42.390516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d67b4fc4-fec1-4a35-bc0d-c3cbd47d1798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:42.441288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063280b1-c792-4dc9-b9a4-bece469e6f2b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.455927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 281/564] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:42.457280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97245c-d055-42b7-9870-6cb8240d4016 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.457595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97245c-d055-42b7-9870-6cb8240d4016 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.457864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97245c-d055-42b7-9870-6cb8240d4016 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.459271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 282/565] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:42.459907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97245c-d055-42b7-9870-6cb8240d4016 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.467606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:42.480478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462353ae-50e0-47ab-b0db-05f680e2ff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:42.481212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462353ae-50e0-47ab-b0db-05f680e2ff5c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.481495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462353ae-50e0-47ab-b0db-05f680e2ff5c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.481760 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462353ae-50e0-47ab-b0db-05f680e2ff5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.493153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462353ae-50e0-47ab-b0db-05f680e2ff5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:42.494610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462353ae-50e0-47ab-b0db-05f680e2ff5c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-462353ae-50e0-47ab-b0db-05f680e2ff5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:42.511699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:42.512651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:42.513538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:42.513538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:42.513538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:42.513538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:42.513538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 284/566] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:42.513812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462353ae-50e0-47ab-b0db-05f680e2ff5c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.532557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:42.569401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97245c-d055-42b7-9870-6cb8240d4016 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.603577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697ba80-bdf1-4519-b7ba-aaa3c5a48872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:42.604239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697ba80-bdf1-4519-b7ba-aaa3c5a48872 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.604541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697ba80-bdf1-4519-b7ba-aaa3c5a48872 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.604818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697ba80-bdf1-4519-b7ba-aaa3c5a48872 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.605679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697ba80-bdf1-4519-b7ba-aaa3c5a48872 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:42.605991 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697ba80-bdf1-4519-b7ba-aaa3c5a48872 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:42.607431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697ba80-bdf1-4519-b7ba-aaa3c5a48872 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.614154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462353ae-50e0-47ab-b0db-05f680e2ff5c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.625675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 283/567] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:42.627029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466809aa-8a65-4b5b-bc23-3d5b7b6ba0de None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.627321 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466809aa-8a65-4b5b-bc23-3d5b7b6ba0de None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.627651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466809aa-8a65-4b5b-bc23-3d5b7b6ba0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.644528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-466809aa-8a65-4b5b-bc23-3d5b7b6ba0de None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:42.645716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-466809aa-8a65-4b5b-bc23-3d5b7b6ba0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:42.648509 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c260fa4-cc4f-4585-a4fd-aca480bc52d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:42.649198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c260fa4-cc4f-4585-a4fd-aca480bc52d2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.649484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c260fa4-cc4f-4585-a4fd-aca480bc52d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.649757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c260fa4-cc4f-4585-a4fd-aca480bc52d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.650476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c260fa4-cc4f-4585-a4fd-aca480bc52d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:42.650786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c260fa4-cc4f-4585-a4fd-aca480bc52d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:42.652221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c260fa4-cc4f-4585-a4fd-aca480bc52d2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.664144 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 285/568] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:42.664924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c260fa4-cc4f-4585-a4fd-aca480bc52d2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.705670 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:42.716755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 284/569] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:42.717577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466809aa-8a65-4b5b-bc23-3d5b7b6ba0de None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.742012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:42.765404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c260fa4-cc4f-4585-a4fd-aca480bc52d2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.797531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadf6c9-920e-44b9-9e07-abc3e08f9064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:42.798327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadf6c9-920e-44b9-9e07-abc3e08f9064 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.798610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadf6c9-920e-44b9-9e07-abc3e08f9064 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.798858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadf6c9-920e-44b9-9e07-abc3e08f9064 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.809036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadf6c9-920e-44b9-9e07-abc3e08f9064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:42.815656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466809aa-8a65-4b5b-bc23-3d5b7b6ba0de None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5cadf6c9-920e-44b9-9e07-abc3e08f9064 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:42.816684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:42.817631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:42.818457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 286/570] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:42.819140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadf6c9-920e-44b9-9e07-abc3e08f9064 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.851659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73362fe8-3c7f-4f6f-9418-72ad332ef86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:42.852325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73362fe8-3c7f-4f6f-9418-72ad332ef86c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.852610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73362fe8-3c7f-4f6f-9418-72ad332ef86c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.852891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73362fe8-3c7f-4f6f-9418-72ad332ef86c None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.853609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73362fe8-3c7f-4f6f-9418-72ad332ef86c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:42.853918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73362fe8-3c7f-4f6f-9418-72ad332ef86c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:42.855315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73362fe8-3c7f-4f6f-9418-72ad332ef86c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.868116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 285/571] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:42.868803 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73362fe8-3c7f-4f6f-9418-72ad332ef86c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:42.920746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:42.926508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadf6c9-920e-44b9-9e07-abc3e08f9064 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.960434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d66ad-7d96-4976-8588-64dff2b1db4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:42.961224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d66ad-7d96-4976-8588-64dff2b1db4f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:42.961505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d66ad-7d96-4976-8588-64dff2b1db4f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:42.961778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d66ad-7d96-4976-8588-64dff2b1db4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:42.964411 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:42.968997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73362fe8-3c7f-4f6f-9418-72ad332ef86c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:42.973421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d66ad-7d96-4976-8588-64dff2b1db4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:42.974883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d66ad-7d96-4976-8588-64dff2b1db4f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-809d66ad-7d96-4976-8588-64dff2b1db4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:42.991597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:42.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:42.993628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:42.993628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:42.993628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:42.993628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:42.993628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 287/572] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:42.993978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d66ad-7d96-4976-8588-64dff2b1db4f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.005293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f11dc0-d618-42ef-a10e-c9d018a3b006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.006100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f11dc0-d618-42ef-a10e-c9d018a3b006 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.006379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f11dc0-d618-42ef-a10e-c9d018a3b006 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.006649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f11dc0-d618-42ef-a10e-c9d018a3b006 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.014466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f11dc0-d618-42ef-a10e-c9d018a3b006 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.015942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f11dc0-d618-42ef-a10e-c9d018a3b006 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-43f11dc0-d618-42ef-a10e-c9d018a3b006 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:43.030989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:43.032140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 286/573] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:43.033439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f11dc0-d618-42ef-a10e-c9d018a3b006 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.093992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d66ad-7d96-4976-8588-64dff2b1db4f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.127976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292457e-b643-4f10-8e04-30805f1ab9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.129215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292457e-b643-4f10-8e04-30805f1ab9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.129588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292457e-b643-4f10-8e04-30805f1ab9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.129868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292457e-b643-4f10-8e04-30805f1ab9c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.138298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f11dc0-d618-42ef-a10e-c9d018a3b006 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.140172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292457e-b643-4f10-8e04-30805f1ab9c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9292457e-b643-4f10-8e04-30805f1ab9c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:43.147836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:43.148932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:43.149958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:43.151728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 288/574] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:43.152925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292457e-b643-4f10-8e04-30805f1ab9c1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.153432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:43.174025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d72f-c56d-4e32-8643-ed3f656915a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.174786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d72f-c56d-4e32-8643-ed3f656915a3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.175135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d72f-c56d-4e32-8643-ed3f656915a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.175482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d72f-c56d-4e32-8643-ed3f656915a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.176314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73d72f-c56d-4e32-8643-ed3f656915a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:43.176630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73d72f-c56d-4e32-8643-ed3f656915a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.178005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73d72f-c56d-4e32-8643-ed3f656915a3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.185849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 287/575] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:43.186617 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d72f-c56d-4e32-8643-ed3f656915a3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.200709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:43.250522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292457e-b643-4f10-8e04-30805f1ab9c1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.284135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d72f-c56d-4e32-8643-ed3f656915a3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.285079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa96323-7e4c-462d-b62d-f6a6a6cb831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.285878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa96323-7e4c-462d-b62d-f6a6a6cb831a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.286179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa96323-7e4c-462d-b62d-f6a6a6cb831a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.286438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa96323-7e4c-462d-b62d-f6a6a6cb831a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.293177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa96323-7e4c-462d-b62d-f6a6a6cb831a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.294628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa96323-7e4c-462d-b62d-f6a6a6cb831a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ffa96323-7e4c-462d-b62d-f6a6a6cb831a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:43.308615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:43.309513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:43.310787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:43.310787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:43.310787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:43.310787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:43.310787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:43.310787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:43.310787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:43.310787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:43.310787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 289/576] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:43.311403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa96323-7e4c-462d-b62d-f6a6a6cb831a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.321530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19c536-775d-4f06-85fa-a45f013e6b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.322211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19c536-775d-4f06-85fa-a45f013e6b20 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.322476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19c536-775d-4f06-85fa-a45f013e6b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.322749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19c536-775d-4f06-85fa-a45f013e6b20 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.323798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19c536-775d-4f06-85fa-a45f013e6b20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:43.324146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19c536-775d-4f06-85fa-a45f013e6b20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.325516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19c536-775d-4f06-85fa-a45f013e6b20 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.337540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 288/577] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:43.338301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19c536-775d-4f06-85fa-a45f013e6b20 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.369677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:43.405166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa96323-7e4c-462d-b62d-f6a6a6cb831a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.414638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:43.436058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19c536-775d-4f06-85fa-a45f013e6b20 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.439767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cd3fa-81bf-4fd5-ae08-ab6287554135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.440433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cd3fa-81bf-4fd5-ae08-ab6287554135 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.440720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cd3fa-81bf-4fd5-ae08-ab6287554135 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.440994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cd3fa-81bf-4fd5-ae08-ab6287554135 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.441822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cd3fa-81bf-4fd5-ae08-ab6287554135 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:43.442351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cd3fa-81bf-4fd5-ae08-ab6287554135 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.443587 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cd3fa-81bf-4fd5-ae08-ab6287554135 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.460258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 290/578] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:43.461055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cd3fa-81bf-4fd5-ae08-ab6287554135 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.474962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfbd59-7232-4faa-afe9-b0298e5753fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.475658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfbd59-7232-4faa-afe9-b0298e5753fa None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.475940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfbd59-7232-4faa-afe9-b0298e5753fa None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.476214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfbd59-7232-4faa-afe9-b0298e5753fa None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.476955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfbd59-7232-4faa-afe9-b0298e5753fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:43.477566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfbd59-7232-4faa-afe9-b0298e5753fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.478620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfbd59-7232-4faa-afe9-b0298e5753fa None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.486927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 289/579] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:43.487746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfbd59-7232-4faa-afe9-b0298e5753fa None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.559432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cd3fa-81bf-4fd5-ae08-ab6287554135 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.579376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:43.586405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfbd59-7232-4faa-afe9-b0298e5753fa None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.592992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a618eb-ac58-4abc-a86d-907fa86624e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.593796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a618eb-ac58-4abc-a86d-907fa86624e4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.594086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a618eb-ac58-4abc-a86d-907fa86624e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.594366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a618eb-ac58-4abc-a86d-907fa86624e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.604316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a618eb-ac58-4abc-a86d-907fa86624e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.605840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a618eb-ac58-4abc-a86d-907fa86624e4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.618807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cdcea-993a-4212-8a5e-8996ffeeb229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.619474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cdcea-993a-4212-8a5e-8996ffeeb229 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.619764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cdcea-993a-4212-8a5e-8996ffeeb229 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.620034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cdcea-993a-4212-8a5e-8996ffeeb229 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.620813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cdcea-993a-4212-8a5e-8996ffeeb229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:43.621150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cdcea-993a-4212-8a5e-8996ffeeb229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.622556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cdcea-993a-4212-8a5e-8996ffeeb229 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-49a618eb-ac58-4abc-a86d-907fa86624e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:43.622864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:43.623734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:43.624675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:43.624675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:43.624675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:43.624675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:43.624675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 291/580] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:43.625239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a618eb-ac58-4abc-a86d-907fa86624e4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.638766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 290/581] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:43.639580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cdcea-993a-4212-8a5e-8996ffeeb229 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.645778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:43.726994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a618eb-ac58-4abc-a86d-907fa86624e4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.733728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cdcea-993a-4212-8a5e-8996ffeeb229 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.759686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020d2d2-de28-4363-85f5-e268d65c7259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.760479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020d2d2-de28-4363-85f5-e268d65c7259 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.760761 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020d2d2-de28-4363-85f5-e268d65c7259 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.761037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020d2d2-de28-4363-85f5-e268d65c7259 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.766622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d35a9a-3e33-4a6d-af07-f517660e2f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.767422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d35a9a-3e33-4a6d-af07-f517660e2f0b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.767734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d35a9a-3e33-4a6d-af07-f517660e2f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.767825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020d2d2-de28-4363-85f5-e268d65c7259 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.768012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d35a9a-3e33-4a6d-af07-f517660e2f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.769200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020d2d2-de28-4363-85f5-e268d65c7259 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.777894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d35a9a-3e33-4a6d-af07-f517660e2f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.779328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d35a9a-3e33-4a6d-af07-f517660e2f0b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c020d2d2-de28-4363-85f5-e268d65c7259 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:43.783063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:43.783945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:43.784800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:43.784800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:43.784800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:43.784800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:43.784800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:43.784800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:43.784800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:43.784800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:43.784800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 292/582] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:43.785274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020d2d2-de28-4363-85f5-e268d65c7259 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-14d35a9a-3e33-4a6d-af07-f517660e2f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:43.800116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:43.801120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:43.802132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:43.802132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:43.802132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:43.802132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:43.802132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 291/583] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:43.802132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d35a9a-3e33-4a6d-af07-f517660e2f0b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.813791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:43.869393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:43.890699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020d2d2-de28-4363-85f5-e268d65c7259 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.898726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d35a9a-3e33-4a6d-af07-f517660e2f0b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:43.923843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c090c8-c37e-46fc-b508-e1f35d34c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.924957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c090c8-c37e-46fc-b508-e1f35d34c412 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.925246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c090c8-c37e-46fc-b508-e1f35d34c412 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.925524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c090c8-c37e-46fc-b508-e1f35d34c412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.932214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd10e6c-55f6-47cc-90ba-9d7d9b29b4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:43.932872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd10e6c-55f6-47cc-90ba-9d7d9b29b4fe None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:43.933154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd10e6c-55f6-47cc-90ba-9d7d9b29b4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:43.933534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd10e6c-55f6-47cc-90ba-9d7d9b29b4fe None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:43.934241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd10e6c-55f6-47cc-90ba-9d7d9b29b4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:43.934547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd10e6c-55f6-47cc-90ba-9d7d9b29b4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.935954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c090c8-c37e-46fc-b508-e1f35d34c412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:43.936073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd10e6c-55f6-47cc-90ba-9d7d9b29b4fe None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.937500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c090c8-c37e-46fc-b508-e1f35d34c412 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:43.947855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 292/584] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:43.948947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd10e6c-55f6-47cc-90ba-9d7d9b29b4fe None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f4c090c8-c37e-46fc-b508-e1f35d34c412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:43.955599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:43.956507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:43.957382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:43.957382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:43.957382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:43.957382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:43.957382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 293/585] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:43.957652 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c090c8-c37e-46fc-b508-e1f35d34c412 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.014876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:44.049038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd10e6c-55f6-47cc-90ba-9d7d9b29b4fe None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.061276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c090c8-c37e-46fc-b508-e1f35d34c412 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.083928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fdad5-e330-4cdb-bb6b-795c4dbbdbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.084587 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fdad5-e330-4cdb-bb6b-795c4dbbdbfa None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.084876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fdad5-e330-4cdb-bb6b-795c4dbbdbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.085153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fdad5-e330-4cdb-bb6b-795c4dbbdbfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.086275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fdad5-e330-4cdb-bb6b-795c4dbbdbfa None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=3f27793f58c74ffb9a7a8e4cdddb8169) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:44.086594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fdad5-e330-4cdb-bb6b-795c4dbbdbfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.088074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fdad5-e330-4cdb-bb6b-795c4dbbdbfa None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.088579 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:44.094061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 293/586] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/role_assignments?user.id=3f27793f58c74ffb9a7a8e4cdddb8169&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:44.094870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fdad5-e330-4cdb-bb6b-795c4dbbdbfa None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.094997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a146-b3ff-48ef-a06e-a5d2e924584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.095566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a146-b3ff-48ef-a06e-a5d2e924584c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.095950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a146-b3ff-48ef-a06e-a5d2e924584c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.096223 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a146-b3ff-48ef-a06e-a5d2e924584c None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.097014 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a146-b3ff-48ef-a06e-a5d2e924584c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:44.097309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a146-b3ff-48ef-a06e-a5d2e924584c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.098774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a146-b3ff-48ef-a06e-a5d2e924584c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.112679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 294/587] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:44.113748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a146-b3ff-48ef-a06e-a5d2e924584c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.206520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fdad5-e330-4cdb-bb6b-795c4dbbdbfa None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.230871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a146-b3ff-48ef-a06e-a5d2e924584c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.238238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:44.245716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c487583-4f61-4c74-8e6e-01a962a02841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.246755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c487583-4f61-4c74-8e6e-01a962a02841 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.247111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c487583-4f61-4c74-8e6e-01a962a02841 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.247419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c487583-4f61-4c74-8e6e-01a962a02841 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.248179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c487583-4f61-4c74-8e6e-01a962a02841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:44.249014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c487583-4f61-4c74-8e6e-01a962a02841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.249847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c487583-4f61-4c74-8e6e-01a962a02841 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.262202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 294/588] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:44.262948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c487583-4f61-4c74-8e6e-01a962a02841 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.270684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834a4fd-1293-4f97-90f2-472ea6fd0b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.271974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834a4fd-1293-4f97-90f2-472ea6fd0b30 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.272293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834a4fd-1293-4f97-90f2-472ea6fd0b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.272596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834a4fd-1293-4f97-90f2-472ea6fd0b30 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.282323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834a4fd-1293-4f97-90f2-472ea6fd0b30 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.284544 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834a4fd-1293-4f97-90f2-472ea6fd0b30 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9834a4fd-1293-4f97-90f2-472ea6fd0b30 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:44.303540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:44.304500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:44.305499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 295/589] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:44.306088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834a4fd-1293-4f97-90f2-472ea6fd0b30 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.358026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:44.377935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c487583-4f61-4c74-8e6e-01a962a02841 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.411681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834a4fd-1293-4f97-90f2-472ea6fd0b30 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.414506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04437c-210d-48c4-b8fd-3c1d8ec7a3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.414638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04437c-210d-48c4-b8fd-3c1d8ec7a3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.414856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04437c-210d-48c4-b8fd-3c1d8ec7a3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.415134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04437c-210d-48c4-b8fd-3c1d8ec7a3c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.415895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04437c-210d-48c4-b8fd-3c1d8ec7a3c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:44.416209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04437c-210d-48c4-b8fd-3c1d8ec7a3c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.417582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04437c-210d-48c4-b8fd-3c1d8ec7a3c7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.430448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 295/590] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:44.431105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04437c-210d-48c4-b8fd-3c1d8ec7a3c7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.447319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15a2e0-bff4-4695-9fb0-9839e5326fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.448256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15a2e0-bff4-4695-9fb0-9839e5326fba None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.448866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15a2e0-bff4-4695-9fb0-9839e5326fba None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.449208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15a2e0-bff4-4695-9fb0-9839e5326fba None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/e80bb5b10ce945cabbfb88ca5df92e3c/roles/ee6ccebaaadd458b9d6b4397f2355b3c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.463636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15a2e0-bff4-4695-9fb0-9839e5326fba None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=e80bb5b10ce945cabbfb88ca5df92e3c, role_id=ee6ccebaaadd458b9d6b4397f2355b3c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.465149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15a2e0-bff4-4695-9fb0-9839e5326fba None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.479340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 296/591] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:28:44 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/e80bb5b10ce945cabbfb88ca5df92e3c/roles/ee6ccebaaadd458b9d6b4397f2355b3c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:44.480192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15a2e0-bff4-4695-9fb0-9839e5326fba None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.499034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:44.560280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04437c-210d-48c4-b8fd-3c1d8ec7a3c7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.604358 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:44.608527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15a2e0-bff4-4695-9fb0-9839e5326fba None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.613479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fefd6-7e5a-45d8-9bf5-55829427018b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.614482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fefd6-7e5a-45d8-9bf5-55829427018b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.614768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fefd6-7e5a-45d8-9bf5-55829427018b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.615073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fefd6-7e5a-45d8-9bf5-55829427018b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.626145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fefd6-7e5a-45d8-9bf5-55829427018b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.627691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fefd6-7e5a-45d8-9bf5-55829427018b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-782fefd6-7e5a-45d8-9bf5-55829427018b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:44.644651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:44.645630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:44.646506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:44.646506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:44.646506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:44.646506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:44.646506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:44.646506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:44.646506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:44.646506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:44.646506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 296/592] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:44.647608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fefd6-7e5a-45d8-9bf5-55829427018b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.661357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a325e-e293-46a8-ba6c-6deb8c29cfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.662692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a325e-e293-46a8-ba6c-6deb8c29cfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.663251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a325e-e293-46a8-ba6c-6deb8c29cfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.663610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a325e-e293-46a8-ba6c-6deb8c29cfe2 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.664457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a325e-e293-46a8-ba6c-6deb8c29cfe2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:44.664770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a325e-e293-46a8-ba6c-6deb8c29cfe2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.666214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a325e-e293-46a8-ba6c-6deb8c29cfe2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.685149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 297/593] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 13 20:28:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:44.688314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a325e-e293-46a8-ba6c-6deb8c29cfe2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.764113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fefd6-7e5a-45d8-9bf5-55829427018b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.777702 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:44.796622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a325e-e293-46a8-ba6c-6deb8c29cfe2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:44.812615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7a91a-3fb6-4e53-a5db-126ed90aa5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.814444 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7a91a-3fb6-4e53-a5db-126ed90aa5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.814963 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7a91a-3fb6-4e53-a5db-126ed90aa5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.815673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7a91a-3fb6-4e53-a5db-126ed90aa5c3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.825940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7a91a-3fb6-4e53-a5db-126ed90aa5c3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.828207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7a91a-3fb6-4e53-a5db-126ed90aa5c3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.836913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df503db1-7bd6-43c1-b318-45bf5eba2248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:44.837972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df503db1-7bd6-43c1-b318-45bf5eba2248 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:44.838329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df503db1-7bd6-43c1-b318-45bf5eba2248 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:44.838674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df503db1-7bd6-43c1-b318-45bf5eba2248 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:44.839577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df503db1-7bd6-43c1-b318-45bf5eba2248 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:44.840026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df503db1-7bd6-43c1-b318-45bf5eba2248 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:44.841564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df503db1-7bd6-43c1-b318-45bf5eba2248 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-cfd7a91a-3fb6-4e53-a5db-126ed90aa5c3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:44.848012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:44.848930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:28:44.849848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:44.857230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 297/594] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:44.857230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7a91a-3fb6-4e53-a5db-126ed90aa5c3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.861328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 298/595] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:44.863078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df503db1-7bd6-43c1-b318-45bf5eba2248 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:44.886880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:44.991394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df503db1-7bd6-43c1-b318-45bf5eba2248 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:45.004765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7a91a-3fb6-4e53-a5db-126ed90aa5c3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:45.037285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2fdd-a75c-40ab-98c0-3abd557fcbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:45.038127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2fdd-a75c-40ab-98c0-3abd557fcbba None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:45.038511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2fdd-a75c-40ab-98c0-3abd557fcbba None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:45.038892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2fdd-a75c-40ab-98c0-3abd557fcbba None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:45.039869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2fdd-a75c-40ab-98c0-3abd557fcbba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:45.040280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2fdd-a75c-40ab-98c0-3abd557fcbba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:45.041502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac1094-5a5c-49a6-8f0a-93e6fc2c7653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:45.041939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2fdd-a75c-40ab-98c0-3abd557fcbba None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:45.042624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac1094-5a5c-49a6-8f0a-93e6fc2c7653 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:45.043102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac1094-5a5c-49a6-8f0a-93e6fc2c7653 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:45.043554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac1094-5a5c-49a6-8f0a-93e6fc2c7653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:45.058189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac1094-5a5c-49a6-8f0a-93e6fc2c7653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:45.065113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac1094-5a5c-49a6-8f0a-93e6fc2c7653 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:45.070587 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 299/596] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Jul 13 20:28:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:45.070587 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2fdd-a75c-40ab-98c0-3abd557fcbba None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-21ac1094-5a5c-49a6-8f0a-93e6fc2c7653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:45.089462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:45.090614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:45.091728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:45.091728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:45.091728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:45.091728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:45.093788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 298/597] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:45.094847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac1094-5a5c-49a6-8f0a-93e6fc2c7653 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:45.122657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:45.174577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:45.200172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2fdd-a75c-40ab-98c0-3abd557fcbba None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:45.205622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac1094-5a5c-49a6-8f0a-93e6fc2c7653 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:45.238110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2e633-a6c0-464d-b395-0ddd87cac096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:45.238958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2e633-a6c0-464d-b395-0ddd87cac096 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:45.239344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2e633-a6c0-464d-b395-0ddd87cac096 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:45.239760 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2e633-a6c0-464d-b395-0ddd87cac096 None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/cc21faf2237d405bba03e5309d93629f/roles/ee6ccebaaadd458b9d6b4397f2355b3c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:45.243910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad22109-9fd3-4e23-81f4-c1280a8c097a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:45.244941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad22109-9fd3-4e23-81f4-c1280a8c097a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:45.245367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad22109-9fd3-4e23-81f4-c1280a8c097a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:45.245861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad22109-9fd3-4e23-81f4-c1280a8c097a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:45.255660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2e633-a6c0-464d-b395-0ddd87cac096 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=cc21faf2237d405bba03e5309d93629f, role_id=ee6ccebaaadd458b9d6b4397f2355b3c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:45.257267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2e633-a6c0-464d-b395-0ddd87cac096 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:45.257873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad22109-9fd3-4e23-81f4-c1280a8c097a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:45.259643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad22109-9fd3-4e23-81f4-c1280a8c097a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:45.272172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 300/598] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:28:45 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/cc21faf2237d405bba03e5309d93629f/roles/ee6ccebaaadd458b9d6b4397f2355b3c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:45.273141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2e633-a6c0-464d-b395-0ddd87cac096 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-aad22109-9fd3-4e23-81f4-c1280a8c097a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:45.279250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:45.280357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:45.281248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:45.281248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:45.281248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:45.281248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:45.287771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 299/599] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:45.289451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad22109-9fd3-4e23-81f4-c1280a8c097a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:45.377634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:45.394748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2e633-a6c0-464d-b395-0ddd87cac096 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:45.403391 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad22109-9fd3-4e23-81f4-c1280a8c097a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:45.426054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:45.438054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa0398-c631-4f54-af2b-e596377ee5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:45.439073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa0398-c631-4f54-af2b-e596377ee5cd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:45.439596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa0398-c631-4f54-af2b-e596377ee5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:45.440076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa0398-c631-4f54-af2b-e596377ee5cd None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:45.440760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a2e0d-1c9d-4dbb-8749-00881e547b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:45.441296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa0398-c631-4f54-af2b-e596377ee5cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:45.441419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a2e0d-1c9d-4dbb-8749-00881e547b10 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:45.441707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a2e0d-1c9d-4dbb-8749-00881e547b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:45.441988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a2e0d-1c9d-4dbb-8749-00881e547b10 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:45.442319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa0398-c631-4f54-af2b-e596377ee5cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:45.442766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a2e0d-1c9d-4dbb-8749-00881e547b10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:45.443081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a2e0d-1c9d-4dbb-8749-00881e547b10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:45.444382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa0398-c631-4f54-af2b-e596377ee5cd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:45.444506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a2e0d-1c9d-4dbb-8749-00881e547b10 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:45.459962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 301/601] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:45.465138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 300/601] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:45.465317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa0398-c631-4f54-af2b-e596377ee5cd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:45.466168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a2e0d-1c9d-4dbb-8749-00881e547b10 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:45.597849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a2e0d-1c9d-4dbb-8749-00881e547b10 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:45.604858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa0398-c631-4f54-af2b-e596377ee5cd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:45.649993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f76cc0-6718-481a-986f-1a98af028bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:45.650729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f76cc0-6718-481a-986f-1a98af028bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:45.651016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f76cc0-6718-481a-986f-1a98af028bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:45.651291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f76cc0-6718-481a-986f-1a98af028bd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:45.664854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f76cc0-6718-481a-986f-1a98af028bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:45.667927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4142a00-5ef5-4916-8605-fba751d1ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:45.668759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4142a00-5ef5-4916-8605-fba751d1ebaf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:45.669044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4142a00-5ef5-4916-8605-fba751d1ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:45.669320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4142a00-5ef5-4916-8605-fba751d1ebaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:45.670443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f76cc0-6718-481a-986f-1a98af028bd0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:45.686540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4142a00-5ef5-4916-8605-fba751d1ebaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:45.688064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4142a00-5ef5-4916-8605-fba751d1ebaf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:45.695787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-17f76cc0-6718-481a-986f-1a98af028bd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:45.703201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:45.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:45.704995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:45.704995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:45.704995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:45.704995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:45.704995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:45.704995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:45.704995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:45.704995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:45.704995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 302/602] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:45.710229 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f76cc0-6718-481a-986f-1a98af028bd0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e4142a00-5ef5-4916-8605-fba751d1ebaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:45.717171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:45.720419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:45.722638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:45.722638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:45.722638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:45.722638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:45.722638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:45.722638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:45.722638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:45.722638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:45.723654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 301/603] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:45.725052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4142a00-5ef5-4916-8605-fba751d1ebaf None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:45.802668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:45.887047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4142a00-5ef5-4916-8605-fba751d1ebaf None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:45.923497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de30a0-91e7-4158-ad1d-89e8e43e576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:45.924394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de30a0-91e7-4158-ad1d-89e8e43e576c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:45.924823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de30a0-91e7-4158-ad1d-89e8e43e576c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:45.925268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de30a0-91e7-4158-ad1d-89e8e43e576c None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:45.926654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de30a0-91e7-4158-ad1d-89e8e43e576c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:45.927110 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de30a0-91e7-4158-ad1d-89e8e43e576c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:45.928741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de30a0-91e7-4158-ad1d-89e8e43e576c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:45.947556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 302/604] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:45.950231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de30a0-91e7-4158-ad1d-89e8e43e576c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:45.962857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f76cc0-6718-481a-986f-1a98af028bd0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:46.023852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:46.069210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de30a0-91e7-4158-ad1d-89e8e43e576c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:46.105326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e51a4-08bb-4c3d-84c8-7b1b909dd3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:46.105580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf7e3a-087f-4c90-a8e6-fd0de214e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:46.106312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf7e3a-087f-4c90-a8e6-fd0de214e4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:46.106699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf7e3a-087f-4c90-a8e6-fd0de214e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:46.106929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e51a4-08bb-4c3d-84c8-7b1b909dd3c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:46.107160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf7e3a-087f-4c90-a8e6-fd0de214e4b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:46.107561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e51a4-08bb-4c3d-84c8-7b1b909dd3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:46.107976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e51a4-08bb-4c3d-84c8-7b1b909dd3c5 None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/9fc23c141eff42f98d31deb2d8aaaca6/roles/ee6ccebaaadd458b9d6b4397f2355b3c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:46.108263 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf7e3a-087f-4c90-a8e6-fd0de214e4b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:46.108670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf7e3a-087f-4c90-a8e6-fd0de214e4b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:46.112436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf7e3a-087f-4c90-a8e6-fd0de214e4b4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:46.138502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e51a4-08bb-4c3d-84c8-7b1b909dd3c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=9fc23c141eff42f98d31deb2d8aaaca6, role_id=ee6ccebaaadd458b9d6b4397f2355b3c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:46.140008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 303/605] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:46.141738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e51a4-08bb-4c3d-84c8-7b1b909dd3c5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:46.145286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf7e3a-087f-4c90-a8e6-fd0de214e4b4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:46.178075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 303/606] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:28:45 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/9fc23c141eff42f98d31deb2d8aaaca6/roles/ee6ccebaaadd458b9d6b4397f2355b3c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:46.279999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf7e3a-087f-4c90-a8e6-fd0de214e4b4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:46.287821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:46.316945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937665e-c007-41d3-a536-ce64b05bffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:46.318142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937665e-c007-41d3-a536-ce64b05bffa4 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:46.318586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937665e-c007-41d3-a536-ce64b05bffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:46.319005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937665e-c007-41d3-a536-ce64b05bffa4 None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/c57d44c19d15496e96c6e37510ce67fc/roles/ee6ccebaaadd458b9d6b4397f2355b3c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:46.336614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937665e-c007-41d3-a536-ce64b05bffa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=c57d44c19d15496e96c6e37510ce67fc, role_id=ee6ccebaaadd458b9d6b4397f2355b3c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:46.338378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937665e-c007-41d3-a536-ce64b05bffa4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:46.357590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c19cc6-a0e9-4baa-ae97-fca29a173c24 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:46.358146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c19cc6-a0e9-4baa-ae97-fca29a173c24 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:46.358567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c19cc6-a0e9-4baa-ae97-fca29a173c24 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:46.360215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 304/607] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:46.364947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 304/608] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:28:46 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/c57d44c19d15496e96c6e37510ce67fc/roles/ee6ccebaaadd458b9d6b4397f2355b3c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:28:46.374044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9f398-efbe-498a-89b7-78f18757e9b8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:46.374714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9f398-efbe-498a-89b7-78f18757e9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:46.375178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9f398-efbe-498a-89b7-78f18757e9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:46.395051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f6d9f398-efbe-498a-89b7-78f18757e9b8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:46.396754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f6d9f398-efbe-498a-89b7-78f18757e9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:46.473118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 305/609] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:46.497091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937665e-c007-41d3-a536-ce64b05bffa4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:46.607518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937665e-c007-41d3-a536-ce64b05bffa4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:46.642594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b55fb-172b-4432-8b17-97605c3b5e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:46.643859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b55fb-172b-4432-8b17-97605c3b5e26 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:46.644359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b55fb-172b-4432-8b17-97605c3b5e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:46.644864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b55fb-172b-4432-8b17-97605c3b5e26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:46.656209 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b55fb-172b-4432-8b17-97605c3b5e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-626b55fb-172b-4432-8b17-97605c3b5e26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:46.664370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:46.665338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:46.666170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:46.669272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 305/610] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:46.674620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a13751-8cce-49df-8dde-6356bd1f4821 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:46.675080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a13751-8cce-49df-8dde-6356bd1f4821 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:46.675296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a13751-8cce-49df-8dde-6356bd1f4821 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:46.676203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b55fb-172b-4432-8b17-97605c3b5e26 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:46.676875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 306/611] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:46.684158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fef68-0528-4823-89d4-a8c060841850 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:46.684452 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fef68-0528-4823-89d4-a8c060841850 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:46.684736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fef68-0528-4823-89d4-a8c060841850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:46.706581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:46.714985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1f9fef68-0528-4823-89d4-a8c060841850 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:46.717850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1f9fef68-0528-4823-89d4-a8c060841850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:46.807912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:46.828692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 307/612] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:46.870360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fef68-0528-4823-89d4-a8c060841850 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:46.882812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b55fb-172b-4432-8b17-97605c3b5e26 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:46.973313 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fef68-0528-4823-89d4-a8c060841850 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:46.973541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0a0ea-0263-430c-8304-2be9951af89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:46.974184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0a0ea-0263-430c-8304-2be9951af89f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:46.974479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0a0ea-0263-430c-8304-2be9951af89f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:46.974760 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0a0ea-0263-430c-8304-2be9951af89f None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:46.975550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0a0ea-0263-430c-8304-2be9951af89f None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:46.975870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0a0ea-0263-430c-8304-2be9951af89f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:46.977284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0a0ea-0263-430c-8304-2be9951af89f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:47.001730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 306/613] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:47.007564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0a0ea-0263-430c-8304-2be9951af89f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:47.010112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61097f2-a642-4153-add4-5439258bbda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:47.010921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61097f2-a642-4153-add4-5439258bbda4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.011209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61097f2-a642-4153-add4-5439258bbda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.013277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61097f2-a642-4153-add4-5439258bbda4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.033889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61097f2-a642-4153-add4-5439258bbda4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f61097f2-a642-4153-add4-5439258bbda4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:47.045068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:47.046010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:47.046855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:47.048692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 308/614] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:47.061560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61097f2-a642-4153-add4-5439258bbda4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:47.096018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:47.174309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61097f2-a642-4153-add4-5439258bbda4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:47.178776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:47.213883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909114d5-3ce4-435a-8de6-fc4f1a8549bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:47.214982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909114d5-3ce4-435a-8de6-fc4f1a8549bc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.215481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909114d5-3ce4-435a-8de6-fc4f1a8549bc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.215898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909114d5-3ce4-435a-8de6-fc4f1a8549bc None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.216853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909114d5-3ce4-435a-8de6-fc4f1a8549bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:47.217302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909114d5-3ce4-435a-8de6-fc4f1a8549bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:47.218842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909114d5-3ce4-435a-8de6-fc4f1a8549bc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:47.226149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0a0ea-0263-430c-8304-2be9951af89f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:47.228249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 309/615] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:47.238418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909114d5-3ce4-435a-8de6-fc4f1a8549bc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:47.293300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dcd40-e21e-4aee-ae1f-1de034abf393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:47.294114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dcd40-e21e-4aee-ae1f-1de034abf393 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.294420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dcd40-e21e-4aee-ae1f-1de034abf393 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.294673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dcd40-e21e-4aee-ae1f-1de034abf393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.316585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3dcd40-e21e-4aee-ae1f-1de034abf393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:47.318104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3dcd40-e21e-4aee-ae1f-1de034abf393 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:47.336177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:47.344990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909114d5-3ce4-435a-8de6-fc4f1a8549bc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-fb3dcd40-e21e-4aee-ae1f-1de034abf393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:47.354858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:47.356003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:47.357096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:47.357096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:47.357096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:47.357096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:47.357096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 307/616] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:47.362644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dcd40-e21e-4aee-ae1f-1de034abf393 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:47.388284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c597ea-7566-486f-9fd6-c35b41327237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:47.390528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c597ea-7566-486f-9fd6-c35b41327237 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.391112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c597ea-7566-486f-9fd6-c35b41327237 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.392275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c597ea-7566-486f-9fd6-c35b41327237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.414704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c597ea-7566-486f-9fd6-c35b41327237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:47.417189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c597ea-7566-486f-9fd6-c35b41327237 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e6c597ea-7566-486f-9fd6-c35b41327237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:47.450962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:47.452119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:47.453147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:47.453147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:47.453147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:47.453147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:47.455993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 310/617] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:47.461508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c597ea-7566-486f-9fd6-c35b41327237 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:47.569692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c597ea-7566-486f-9fd6-c35b41327237 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:47.570316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dcd40-e21e-4aee-ae1f-1de034abf393 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:47.609383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea599f74-f0e4-4a0a-9cb6-a7a84ed25ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:47.610304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599f74-f0e4-4a0a-9cb6-a7a84ed25ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.610735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599f74-f0e4-4a0a-9cb6-a7a84ed25ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.611283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599f74-f0e4-4a0a-9cb6-a7a84ed25ba3 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.613173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea599f74-f0e4-4a0a-9cb6-a7a84ed25ba3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:47.613802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea599f74-f0e4-4a0a-9cb6-a7a84ed25ba3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:47.616420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea599f74-f0e4-4a0a-9cb6-a7a84ed25ba3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:47.629839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:47.632605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:47.639146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 311/618] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:47.640940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16deedf3-442d-42a6-a175-8ca8c60e0853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:47.641604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16deedf3-442d-42a6-a175-8ca8c60e0853 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.641890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16deedf3-442d-42a6-a175-8ca8c60e0853 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.642166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16deedf3-442d-42a6-a175-8ca8c60e0853 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.642893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16deedf3-442d-42a6-a175-8ca8c60e0853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:47.643202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16deedf3-442d-42a6-a175-8ca8c60e0853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:47.645097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea599f74-f0e4-4a0a-9cb6-a7a84ed25ba3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:47.648550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16deedf3-442d-42a6-a175-8ca8c60e0853 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:47.670360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 308/619] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:47.676523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16deedf3-442d-42a6-a175-8ca8c60e0853 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:47.809931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16deedf3-442d-42a6-a175-8ca8c60e0853 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:47.820703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea599f74-f0e4-4a0a-9cb6-a7a84ed25ba3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:47.862686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28288f97-ca07-4148-9a80-96b0611b70f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:47.865165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28288f97-ca07-4148-9a80-96b0611b70f7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.865663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28288f97-ca07-4148-9a80-96b0611b70f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.867066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28288f97-ca07-4148-9a80-96b0611b70f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.878495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bb49a-0024-435a-8c77-3e36ff404481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:47.879615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb49a-0024-435a-8c77-3e36ff404481 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.879973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28288f97-ca07-4148-9a80-96b0611b70f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:47.880369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb49a-0024-435a-8c77-3e36ff404481 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.880858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb49a-0024-435a-8c77-3e36ff404481 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.881475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28288f97-ca07-4148-9a80-96b0611b70f7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:47.889252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bb49a-0024-435a-8c77-3e36ff404481 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:47.890955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bb49a-0024-435a-8c77-3e36ff404481 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-28288f97-ca07-4148-9a80-96b0611b70f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:47.896521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:47.897453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:28:47.898381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:47.902589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 309/620] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:47.903994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e3943-0723-4c8b-ab7b-8df899dcb333 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.904288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e3943-0723-4c8b-ab7b-8df899dcb333 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.904573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e3943-0723-4c8b-ab7b-8df899dcb333 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.906105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 310/621] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:47.906806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e3943-0723-4c8b-ab7b-8df899dcb333 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d45bb49a-0024-435a-8c77-3e36ff404481 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:47.911874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:47.912783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:28:47.913680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:47.917269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 312/622] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 13 20:28:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:47.918784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cab9ab-0268-4752-9ef0-8542fd11fe39 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:47.919282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cab9ab-0268-4752-9ef0-8542fd11fe39 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:47.919836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cab9ab-0268-4752-9ef0-8542fd11fe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:47.938134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b0cab9ab-0268-4752-9ef0-8542fd11fe39 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:47.939715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b0cab9ab-0268-4752-9ef0-8542fd11fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:47.969000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:48.017083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 313/623] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:48.018334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cab9ab-0268-4752-9ef0-8542fd11fe39 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.030517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e3943-0723-4c8b-ab7b-8df899dcb333 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:48.031789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:48.080495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ea885-2f19-4f59-9597-00050c670dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:48.081118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ea885-2f19-4f59-9597-00050c670dbc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.081408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ea885-2f19-4f59-9597-00050c670dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.081688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ea885-2f19-4f59-9597-00050c670dbc None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.082595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ea885-2f19-4f59-9597-00050c670dbc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:48.082910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ea885-2f19-4f59-9597-00050c670dbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:48.084502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ea885-2f19-4f59-9597-00050c670dbc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:48.106978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 311/624] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 13 20:28:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:48.107960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ea885-2f19-4f59-9597-00050c670dbc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.138586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cab9ab-0268-4752-9ef0-8542fd11fe39 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:48.175203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5cea9-7d9e-4ecf-9e30-a0afae18dabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:48.176323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5cea9-7d9e-4ecf-9e30-a0afae18dabf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.176820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5cea9-7d9e-4ecf-9e30-a0afae18dabf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.177302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5cea9-7d9e-4ecf-9e30-a0afae18dabf None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.178439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5cea9-7d9e-4ecf-9e30-a0afae18dabf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:48.178994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5cea9-7d9e-4ecf-9e30-a0afae18dabf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:48.180725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5cea9-7d9e-4ecf-9e30-a0afae18dabf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:48.199546 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 314/625] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:48.200717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5cea9-7d9e-4ecf-9e30-a0afae18dabf None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.214390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ea885-2f19-4f59-9597-00050c670dbc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:48.224420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:48.250969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c388711-fb06-450e-8bbe-b740ac82a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:48.251893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c388711-fb06-450e-8bbe-b740ac82a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.252179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c388711-fb06-450e-8bbe-b740ac82a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.252454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c388711-fb06-450e-8bbe-b740ac82a5c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.256522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:48.266663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c388711-fb06-450e-8bbe-b740ac82a5c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9c388711-fb06-450e-8bbe-b740ac82a5c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:48.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:48.276204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:48.277075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 312/626] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:48.282244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c388711-fb06-450e-8bbe-b740ac82a5c3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.319331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5cea9-7d9e-4ecf-9e30-a0afae18dabf None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:48.356511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0ed85-e81b-4f93-8ac1-769ed60546b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:48.357855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0ed85-e81b-4f93-8ac1-769ed60546b4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.358356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0ed85-e81b-4f93-8ac1-769ed60546b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.358851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0ed85-e81b-4f93-8ac1-769ed60546b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.370431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0ed85-e81b-4f93-8ac1-769ed60546b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:48.372413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0ed85-e81b-4f93-8ac1-769ed60546b4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d8d0ed85-e81b-4f93-8ac1-769ed60546b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:48.391913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:48.392863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:48.393721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:48.393721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:48.393721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:48.393721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:48.394039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c388711-fb06-450e-8bbe-b740ac82a5c3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:48.397073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 315/627] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:48.398119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0ed85-e81b-4f93-8ac1-769ed60546b4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.435012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc682c8-4911-4721-852f-2732a347e221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:48.435751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc682c8-4911-4721-852f-2732a347e221 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.436042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc682c8-4911-4721-852f-2732a347e221 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.436324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc682c8-4911-4721-852f-2732a347e221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.474420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:48.501806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:48.504688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0ed85-e81b-4f93-8ac1-769ed60546b4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:48.512971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc682c8-4911-4721-852f-2732a347e221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:48.515063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc682c8-4911-4721-852f-2732a347e221 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:48.538460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321e2ab-324c-4f65-bffd-cb9b6b82ae76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:48.539129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321e2ab-324c-4f65-bffd-cb9b6b82ae76 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.539411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321e2ab-324c-4f65-bffd-cb9b6b82ae76 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.539719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321e2ab-324c-4f65-bffd-cb9b6b82ae76 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.540462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321e2ab-324c-4f65-bffd-cb9b6b82ae76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:48.540762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321e2ab-324c-4f65-bffd-cb9b6b82ae76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:48.542121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321e2ab-324c-4f65-bffd-cb9b6b82ae76 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:48.550429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 316/628] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:48.551322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321e2ab-324c-4f65-bffd-cb9b6b82ae76 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.626729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 313/629] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:48.628696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a36c3-1e50-44be-914e-1647c8a5f667 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.629167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a36c3-1e50-44be-914e-1647c8a5f667 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.629622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a36c3-1e50-44be-914e-1647c8a5f667 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.631438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 314/630] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:48.632369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a36c3-1e50-44be-914e-1647c8a5f667 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.659965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321e2ab-324c-4f65-bffd-cb9b6b82ae76 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:48.696361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57101c-049f-412e-aa22-a2c8899d0651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:48.697187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57101c-049f-412e-aa22-a2c8899d0651 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.697482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57101c-049f-412e-aa22-a2c8899d0651 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.698065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57101c-049f-412e-aa22-a2c8899d0651 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.699395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57101c-049f-412e-aa22-a2c8899d0651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:48.699732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57101c-049f-412e-aa22-a2c8899d0651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:48.701119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57101c-049f-412e-aa22-a2c8899d0651 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:48.702511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:48.721598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 317/631] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:48.721598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57101c-049f-412e-aa22-a2c8899d0651 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.739980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a36c3-1e50-44be-914e-1647c8a5f667 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:48.746340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:48.779371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74faeee2-c34d-47b3-817e-b7261f8c909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:48.780204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74faeee2-c34d-47b3-817e-b7261f8c909d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.780491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74faeee2-c34d-47b3-817e-b7261f8c909d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.780766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74faeee2-c34d-47b3-817e-b7261f8c909d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.791406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74faeee2-c34d-47b3-817e-b7261f8c909d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:48.792980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74faeee2-c34d-47b3-817e-b7261f8c909d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-74faeee2-c34d-47b3-817e-b7261f8c909d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:48.811471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:48.812413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:48.813279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:48.813279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:48.813279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:48.813279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:48.813279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 315/632] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:48.814895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121a92-6117-4f0e-8652-564b15621ddc None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.815189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121a92-6117-4f0e-8652-564b15621ddc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.815494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121a92-6117-4f0e-8652-564b15621ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.832057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4c121a92-6117-4f0e-8652-564b15621ddc None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:48.833231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4c121a92-6117-4f0e-8652-564b15621ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:48.833815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57101c-049f-412e-aa22-a2c8899d0651 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:48.870381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184d43-57aa-437b-bf95-2a16d3776a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:48.871217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184d43-57aa-437b-bf95-2a16d3776a67 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:48.871672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184d43-57aa-437b-bf95-2a16d3776a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:48.871849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184d43-57aa-437b-bf95-2a16d3776a67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:48.883860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a184d43-57aa-437b-bf95-2a16d3776a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:48.885284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a184d43-57aa-437b-bf95-2a16d3776a67 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2a184d43-57aa-437b-bf95-2a16d3776a67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:48.903594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:48.904509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:48.910013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:48.910013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:48.910013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:48.910013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:48.910240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 316/633] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:48.910240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c121a92-6117-4f0e-8652-564b15621ddc None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.910400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 318/634] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:48.910400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184d43-57aa-437b-bf95-2a16d3776a67 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:48.953353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:48.984242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:49.003393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c121a92-6117-4f0e-8652-564b15621ddc None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.009441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184d43-57aa-437b-bf95-2a16d3776a67 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.037060 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3277d4-f721-4940-a3b3-7c14ab459f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.037853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3277d4-f721-4940-a3b3-7c14ab459f84 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.038155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3277d4-f721-4940-a3b3-7c14ab459f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.038436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3277d4-f721-4940-a3b3-7c14ab459f84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.043392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08f452-232a-47f8-b9ee-52e4a8d76fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.044340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08f452-232a-47f8-b9ee-52e4a8d76fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.044625 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08f452-232a-47f8-b9ee-52e4a8d76fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.044896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08f452-232a-47f8-b9ee-52e4a8d76fa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.045610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08f452-232a-47f8-b9ee-52e4a8d76fa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:49.045915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08f452-232a-47f8-b9ee-52e4a8d76fa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.046240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3277d4-f721-4940-a3b3-7c14ab459f84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.047337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08f452-232a-47f8-b9ee-52e4a8d76fa8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.048786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3277d4-f721-4940-a3b3-7c14ab459f84 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.064333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 319/635] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:49.064333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccd87f-89b8-4b7a-ae06-a2ca1110b7ab None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.064333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccd87f-89b8-4b7a-ae06-a2ca1110b7ab None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ff3277d4-f721-4940-a3b3-7c14ab459f84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:49.064801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:49.065824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:49.066908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:49.066908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:49.066908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:49.066908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:49.066908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:49.066908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:49.066908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:49.066908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:49.067411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccd87f-89b8-4b7a-ae06-a2ca1110b7ab None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.067411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 320/636] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:49.068051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 317/637] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:49.068161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccd87f-89b8-4b7a-ae06-a2ca1110b7ab None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.069112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3277d4-f721-4940-a3b3-7c14ab459f84 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.165192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3277d4-f721-4940-a3b3-7c14ab459f84 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.167990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccd87f-89b8-4b7a-ae06-a2ca1110b7ab None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.175067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:49.204553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a73d5-2871-45cb-a573-05f585474b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.205405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a73d5-2871-45cb-a573-05f585474b0f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.205686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a73d5-2871-45cb-a573-05f585474b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.205957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed2948-c91d-4bc0-b902-c46b9c9a17c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.206084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a73d5-2871-45cb-a573-05f585474b0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.206618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed2948-c91d-4bc0-b902-c46b9c9a17c0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.206903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed2948-c91d-4bc0-b902-c46b9c9a17c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.207172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed2948-c91d-4bc0-b902-c46b9c9a17c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.207987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed2948-c91d-4bc0-b902-c46b9c9a17c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:49.208289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed2948-c91d-4bc0-b902-c46b9c9a17c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.209658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed2948-c91d-4bc0-b902-c46b9c9a17c0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.211071 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:49.216604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a73d5-2871-45cb-a573-05f585474b0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:49.221569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 321/638] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:49.222643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed2948-c91d-4bc0-b902-c46b9c9a17c0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-054a73d5-2871-45cb-a573-05f585474b0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:49.224340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:49.225212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 318/639] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:49.226550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a73d5-2871-45cb-a573-05f585474b0f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.325405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed2948-c91d-4bc0-b902-c46b9c9a17c0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.325908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a73d5-2871-45cb-a573-05f585474b0f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.358885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad70331-6a33-4bbb-924f-72f738163924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.359227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bed01c-53c1-4409-95a1-73cc04d00493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.359541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad70331-6a33-4bbb-924f-72f738163924 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.359823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad70331-6a33-4bbb-924f-72f738163924 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.360041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bed01c-53c1-4409-95a1-73cc04d00493 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.360154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad70331-6a33-4bbb-924f-72f738163924 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.360328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bed01c-53c1-4409-95a1-73cc04d00493 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.360605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bed01c-53c1-4409-95a1-73cc04d00493 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.360862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad70331-6a33-4bbb-924f-72f738163924 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:49.361172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad70331-6a33-4bbb-924f-72f738163924 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.362598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad70331-6a33-4bbb-924f-72f738163924 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.368529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bed01c-53c1-4409-95a1-73cc04d00493 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.369972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bed01c-53c1-4409-95a1-73cc04d00493 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.374406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 322/640] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:49.375654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022239-e246-44fd-b809-273823c10ab0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.375860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022239-e246-44fd-b809-273823c10ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.376173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0022239-e246-44fd-b809-273823c10ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-14bed01c-53c1-4409-95a1-73cc04d00493 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:49.384343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:49.385540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 319/641] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:49.386546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bed01c-53c1-4409-95a1-73cc04d00493 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.396985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c0022239-e246-44fd-b809-273823c10ab0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:49.398545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c0022239-e246-44fd-b809-273823c10ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:49.419363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:49.449573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:49.467105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 323/642] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:49.467959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022239-e246-44fd-b809-273823c10ab0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.487171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bed01c-53c1-4409-95a1-73cc04d00493 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.521193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edca70a-37e9-4219-8391-c4908f63a850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.521993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edca70a-37e9-4219-8391-c4908f63a850 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.522275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edca70a-37e9-4219-8391-c4908f63a850 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.522551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edca70a-37e9-4219-8391-c4908f63a850 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.529283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edca70a-37e9-4219-8391-c4908f63a850 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.530826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edca70a-37e9-4219-8391-c4908f63a850 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7edca70a-37e9-4219-8391-c4908f63a850 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:49.546927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:49.547850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 320/643] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:49.548822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edca70a-37e9-4219-8391-c4908f63a850 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.571001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0022239-e246-44fd-b809-273823c10ab0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.604286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e493009-eee0-4065-a2c6-bf3bbc24f72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.604971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e493009-eee0-4065-a2c6-bf3bbc24f72d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.605325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e493009-eee0-4065-a2c6-bf3bbc24f72d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.605606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e493009-eee0-4065-a2c6-bf3bbc24f72d None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.606349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e493009-eee0-4065-a2c6-bf3bbc24f72d None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:49.606662 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e493009-eee0-4065-a2c6-bf3bbc24f72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.608092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e493009-eee0-4065-a2c6-bf3bbc24f72d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.620377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 324/644] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:49.620377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e493009-eee0-4065-a2c6-bf3bbc24f72d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.641320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:49.651549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edca70a-37e9-4219-8391-c4908f63a850 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.665539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:49.686557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9140dc-3e03-4917-9008-72e0eb2e9ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.687394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9140dc-3e03-4917-9008-72e0eb2e9ef5 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.687727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9140dc-3e03-4917-9008-72e0eb2e9ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.688010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9140dc-3e03-4917-9008-72e0eb2e9ef5 None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/3f27793f58c74ffb9a7a8e4cdddb8169/roles/ee6ccebaaadd458b9d6b4397f2355b3c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.702893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9140dc-3e03-4917-9008-72e0eb2e9ef5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=3f27793f58c74ffb9a7a8e4cdddb8169, role_id=ee6ccebaaadd458b9d6b4397f2355b3c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.704431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9140dc-3e03-4917-9008-72e0eb2e9ef5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.721039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 321/645] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:28:49 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/3f27793f58c74ffb9a7a8e4cdddb8169/roles/ee6ccebaaadd458b9d6b4397f2355b3c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:28:49.721192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e493009-eee0-4065-a2c6-bf3bbc24f72d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.721844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9140dc-3e03-4917-9008-72e0eb2e9ef5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.767752 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a69b0e-1934-4918-97c8-1a7d9d12ff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.768569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a69b0e-1934-4918-97c8-1a7d9d12ff96 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.768896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a69b0e-1934-4918-97c8-1a7d9d12ff96 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.769182 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a69b0e-1934-4918-97c8-1a7d9d12ff96 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.770056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a69b0e-1934-4918-97c8-1a7d9d12ff96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:49.770415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a69b0e-1934-4918-97c8-1a7d9d12ff96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.771929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a69b0e-1934-4918-97c8-1a7d9d12ff96 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.797419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 325/646] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:49.797419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a69b0e-1934-4918-97c8-1a7d9d12ff96 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.847721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9140dc-3e03-4917-9008-72e0eb2e9ef5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.891561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7898cc-b621-428b-b412-c49a15c9a002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.892370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7898cc-b621-428b-b412-c49a15c9a002 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.893422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7898cc-b621-428b-b412-c49a15c9a002 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.893706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7898cc-b621-428b-b412-c49a15c9a002 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.901527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:49.904670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7898cc-b621-428b-b412-c49a15c9a002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:49.910053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a69b0e-1934-4918-97c8-1a7d9d12ff96 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-6e7898cc-b621-428b-b412-c49a15c9a002 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:49.912914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:49.914051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:49.915076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 322/647] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:49.915992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:49.916077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7898cc-b621-428b-b412-c49a15c9a002 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:49.954004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5219c-9129-4ae4-b735-ca7853b79071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:49.954585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5219c-9129-4ae4-b735-ca7853b79071 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:49.954920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5219c-9129-4ae4-b735-ca7853b79071 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:49.955206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5219c-9129-4ae4-b735-ca7853b79071 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:49.956056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5219c-9129-4ae4-b735-ca7853b79071 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:49.956369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5219c-9129-4ae4-b735-ca7853b79071 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:49.957817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5219c-9129-4ae4-b735-ca7853b79071 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:49.970826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 326/648] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:49.971483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5219c-9129-4ae4-b735-ca7853b79071 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.035356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7898cc-b621-428b-b412-c49a15c9a002 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.072470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb410b2-98eb-47f6-9ae9-13b2bded3afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:50.074044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb410b2-98eb-47f6-9ae9-13b2bded3afe None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:50.074336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb410b2-98eb-47f6-9ae9-13b2bded3afe None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:50.074679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb410b2-98eb-47f6-9ae9-13b2bded3afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:50.081508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5219c-9129-4ae4-b735-ca7853b79071 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.086363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb410b2-98eb-47f6-9ae9-13b2bded3afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:50.087940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb410b2-98eb-47f6-9ae9-13b2bded3afe None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-eeb410b2-98eb-47f6-9ae9-13b2bded3afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:50.105413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:50.106557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:50.112278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:50.112278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:50.112278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:50.112278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:50.112278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 323/649] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:50.112278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb410b2-98eb-47f6-9ae9-13b2bded3afe None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.123652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216fc03-81da-4470-9521-7e104785d0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:50.124395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216fc03-81da-4470-9521-7e104785d0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:50.124736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216fc03-81da-4470-9521-7e104785d0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:50.125080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216fc03-81da-4470-9521-7e104785d0a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:50.157544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:50.167940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:50.201200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216fc03-81da-4470-9521-7e104785d0a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:50.202891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216fc03-81da-4470-9521-7e104785d0a8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:50.219622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb410b2-98eb-47f6-9ae9-13b2bded3afe None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.254540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79421339-8dbb-42c8-9ada-122b2ef3b62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:50.255220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79421339-8dbb-42c8-9ada-122b2ef3b62e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:50.255529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79421339-8dbb-42c8-9ada-122b2ef3b62e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:50.255809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79421339-8dbb-42c8-9ada-122b2ef3b62e None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:50.256552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79421339-8dbb-42c8-9ada-122b2ef3b62e None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:50.256856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79421339-8dbb-42c8-9ada-122b2ef3b62e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:50.258268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79421339-8dbb-42c8-9ada-122b2ef3b62e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:50.272010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 324/650] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:50.272010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79421339-8dbb-42c8-9ada-122b2ef3b62e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.316528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 327/651] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:50.317689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216fc03-81da-4470-9521-7e104785d0a8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.382103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79421339-8dbb-42c8-9ada-122b2ef3b62e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.388805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:50.397056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:50.422077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46bd7b-c5cd-482f-8fca-43684dfe395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:50.422744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46bd7b-c5cd-482f-8fca-43684dfe395a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:50.423030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46bd7b-c5cd-482f-8fca-43684dfe395a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:50.423307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46bd7b-c5cd-482f-8fca-43684dfe395a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:50.424179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46bd7b-c5cd-482f-8fca-43684dfe395a None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=e80bb5b10ce945cabbfb88ca5df92e3c) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:50.424504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46bd7b-c5cd-482f-8fca-43684dfe395a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:50.425967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46bd7b-c5cd-482f-8fca-43684dfe395a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:50.431182 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216fc03-81da-4470-9521-7e104785d0a8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.432275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 325/652] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:50 2022] GET /identity/v3/role_assignments?user.id=e80bb5b10ce945cabbfb88ca5df92e3c&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:50.440133 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46bd7b-c5cd-482f-8fca-43684dfe395a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.478466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6096e0-2fbc-440d-8b02-054d0ef7076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:50.479895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6096e0-2fbc-440d-8b02-054d0ef7076c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:50.480750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6096e0-2fbc-440d-8b02-054d0ef7076c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:50.481197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6096e0-2fbc-440d-8b02-054d0ef7076c None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:50.482118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6096e0-2fbc-440d-8b02-054d0ef7076c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:50.482582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6096e0-2fbc-440d-8b02-054d0ef7076c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:50.485678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6096e0-2fbc-440d-8b02-054d0ef7076c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:50.502834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 328/653] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:50.504024 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6096e0-2fbc-440d-8b02-054d0ef7076c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.562155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46bd7b-c5cd-482f-8fca-43684dfe395a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.607981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bca906-5372-421b-8c7f-0cf0559e28da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:50.609050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bca906-5372-421b-8c7f-0cf0559e28da None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:50.609513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bca906-5372-421b-8c7f-0cf0559e28da None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:50.609943 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bca906-5372-421b-8c7f-0cf0559e28da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:50.622707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bca906-5372-421b-8c7f-0cf0559e28da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:50.624474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bca906-5372-421b-8c7f-0cf0559e28da None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:50.639543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6096e0-2fbc-440d-8b02-054d0ef7076c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c0bca906-5372-421b-8c7f-0cf0559e28da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:50.643530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:50.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:50.645655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:50.645655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:50.645655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:50.645655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:50.645655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 326/654] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:50.645921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bca906-5372-421b-8c7f-0cf0559e28da None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.670416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:50.682199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa7d64-cf20-417f-ac73-eb4803d763dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:50.683007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa7d64-cf20-417f-ac73-eb4803d763dd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:50.683375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa7d64-cf20-417f-ac73-eb4803d763dd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:50.683696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa7d64-cf20-417f-ac73-eb4803d763dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:50.684586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:50.700327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa7d64-cf20-417f-ac73-eb4803d763dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:50.700539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa7d64-cf20-417f-ac73-eb4803d763dd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-6eaa7d64-cf20-417f-ac73-eb4803d763dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:50.719686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:50.720622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:50.721505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:50.721505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:50.721505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:50.721505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:50.721505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 329/655] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:50.722242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa7d64-cf20-417f-ac73-eb4803d763dd None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.777251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bca906-5372-421b-8c7f-0cf0559e28da None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.814948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c16b4-7968-40aa-be64-466f10bfb8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:50.816229 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c16b4-7968-40aa-be64-466f10bfb8ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:50.816736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c16b4-7968-40aa-be64-466f10bfb8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:50.817208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c16b4-7968-40aa-be64-466f10bfb8ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:50.827003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c16b4-7968-40aa-be64-466f10bfb8ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:50.829061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c16b4-7968-40aa-be64-466f10bfb8ac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:50.832874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa7d64-cf20-417f-ac73-eb4803d763dd None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8b1c16b4-7968-40aa-be64-466f10bfb8ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:50.846138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:50.847184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:28:50.848172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:50.851042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 327/656] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:50.856904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c16b4-7968-40aa-be64-466f10bfb8ac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.868043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a62fc-aaee-478e-aee6-ec1bedcbf05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:50.868724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a62fc-aaee-478e-aee6-ec1bedcbf05b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:50.869012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a62fc-aaee-478e-aee6-ec1bedcbf05b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:50.869295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a62fc-aaee-478e-aee6-ec1bedcbf05b None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:50.870249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a62fc-aaee-478e-aee6-ec1bedcbf05b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:50.870367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a62fc-aaee-478e-aee6-ec1bedcbf05b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:50.871826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a62fc-aaee-478e-aee6-ec1bedcbf05b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:50.884556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 330/657] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:50.885450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a62fc-aaee-478e-aee6-ec1bedcbf05b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:50.920664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:50.958331 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:50.970295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c16b4-7968-40aa-be64-466f10bfb8ac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:50.998485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a62fc-aaee-478e-aee6-ec1bedcbf05b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:51.006560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702273a-610a-456e-9320-9001429aa567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.007590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702273a-610a-456e-9320-9001429aa567 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.008095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702273a-610a-456e-9320-9001429aa567 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.008579 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702273a-610a-456e-9320-9001429aa567 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.009538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702273a-610a-456e-9320-9001429aa567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:51.010055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702273a-610a-456e-9320-9001429aa567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.011752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702273a-610a-456e-9320-9001429aa567 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.025281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 328/658] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:51.026481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702273a-610a-456e-9320-9001429aa567 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.038603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde8ee-91de-440e-8a64-da648eb015e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.039563 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde8ee-91de-440e-8a64-da648eb015e3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.039861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde8ee-91de-440e-8a64-da648eb015e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.040147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde8ee-91de-440e-8a64-da648eb015e3 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.041281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dde8ee-91de-440e-8a64-da648eb015e3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:51.041600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dde8ee-91de-440e-8a64-da648eb015e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.043281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dde8ee-91de-440e-8a64-da648eb015e3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.068910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 331/659] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 13 20:28:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:51.068910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde8ee-91de-440e-8a64-da648eb015e3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.149666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702273a-610a-456e-9320-9001429aa567 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:51.170053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:51.177268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde8ee-91de-440e-8a64-da648eb015e3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:51.186714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2111ad-743d-4dd8-bda6-75aeb87575bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.187920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2111ad-743d-4dd8-bda6-75aeb87575bc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.188410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2111ad-743d-4dd8-bda6-75aeb87575bc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.188908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2111ad-743d-4dd8-bda6-75aeb87575bc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.197015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2111ad-743d-4dd8-bda6-75aeb87575bc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.199517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2111ad-743d-4dd8-bda6-75aeb87575bc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-db2111ad-743d-4dd8-bda6-75aeb87575bc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:51.216508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:51.217801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:28:51.218729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:51.219269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5dd20-807e-4551-b200-1bda9632721b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.219269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5dd20-807e-4551-b200-1bda9632721b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.219269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5dd20-807e-4551-b200-1bda9632721b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.219269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5dd20-807e-4551-b200-1bda9632721b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.221721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 329/660] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:51.222835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2111ad-743d-4dd8-bda6-75aeb87575bc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.226263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5dd20-807e-4551-b200-1bda9632721b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.228424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5dd20-807e-4551-b200-1bda9632721b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.237176 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2ab5dd20-807e-4551-b200-1bda9632721b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:51.242404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:51.243292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 332/661] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:51.244818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5dd20-807e-4551-b200-1bda9632721b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.336361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2111ad-743d-4dd8-bda6-75aeb87575bc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:51.349095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5dd20-807e-4551-b200-1bda9632721b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:51.373262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2b41a-182b-4f09-8c05-09475e16530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.374338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2b41a-182b-4f09-8c05-09475e16530c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.374829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2b41a-182b-4f09-8c05-09475e16530c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.375325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2b41a-182b-4f09-8c05-09475e16530c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.384423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13245fc-27fd-4d0d-a545-f24619e8f6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.385104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13245fc-27fd-4d0d-a545-f24619e8f6fe None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.385401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13245fc-27fd-4d0d-a545-f24619e8f6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.385734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13245fc-27fd-4d0d-a545-f24619e8f6fe None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.386628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13245fc-27fd-4d0d-a545-f24619e8f6fe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:51.386944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13245fc-27fd-4d0d-a545-f24619e8f6fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.388418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13245fc-27fd-4d0d-a545-f24619e8f6fe None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.406707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 333/662] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:51.407646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13245fc-27fd-4d0d-a545-f24619e8f6fe None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.422293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:51.450843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2b41a-182b-4f09-8c05-09475e16530c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.452521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2b41a-182b-4f09-8c05-09475e16530c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.464522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:51.511689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13245fc-27fd-4d0d-a545-f24619e8f6fe None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:51.547087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347eb78-3cce-4a80-ae37-0c7e309f0351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.548115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347eb78-3cce-4a80-ae37-0c7e309f0351 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.548604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347eb78-3cce-4a80-ae37-0c7e309f0351 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.549059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347eb78-3cce-4a80-ae37-0c7e309f0351 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.550025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347eb78-3cce-4a80-ae37-0c7e309f0351 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:51.550523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347eb78-3cce-4a80-ae37-0c7e309f0351 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.552197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347eb78-3cce-4a80-ae37-0c7e309f0351 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.561675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 330/663] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:51.562586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2b41a-182b-4f09-8c05-09475e16530c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.565334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 334/664] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:51.566376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347eb78-3cce-4a80-ae37-0c7e309f0351 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.638763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:51.667678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2b41a-182b-4f09-8c05-09475e16530c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:51.671240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347eb78-3cce-4a80-ae37-0c7e309f0351 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:51.679029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:51.707259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080f963-7d80-4f09-8138-456b63693dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.707963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080f963-7d80-4f09-8138-456b63693dcc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.708252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080f963-7d80-4f09-8138-456b63693dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.708533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080f963-7d80-4f09-8138-456b63693dcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.710486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9698-a5b0-4a55-bd54-c78f0db7c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.711560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9698-a5b0-4a55-bd54-c78f0db7c9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.712066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9698-a5b0-4a55-bd54-c78f0db7c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.712556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9698-a5b0-4a55-bd54-c78f0db7c9c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.724181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d9698-a5b0-4a55-bd54-c78f0db7c9c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.725937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d9698-a5b0-4a55-bd54-c78f0db7c9c9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e45d9698-a5b0-4a55-bd54-c78f0db7c9c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:51.744129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:51.745056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:51.745966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:51.745966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:51.745966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:51.745966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:51.748220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 335/665] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:51.749220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9698-a5b0-4a55-bd54-c78f0db7c9c9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.783986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080f963-7d80-4f09-8138-456b63693dcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.785458 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080f963-7d80-4f09-8138-456b63693dcc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.853769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9698-a5b0-4a55-bd54-c78f0db7c9c9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:51.876846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:51.888275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665b068-c90b-4743-a4b1-26e1711bec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:51.888990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665b068-c90b-4743-a4b1-26e1711bec95 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.889231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665b068-c90b-4743-a4b1-26e1711bec95 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.889514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665b068-c90b-4743-a4b1-26e1711bec95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.890350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665b068-c90b-4743-a4b1-26e1711bec95 None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=cc21faf2237d405bba03e5309d93629f) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:51.890669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665b068-c90b-4743-a4b1-26e1711bec95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:51.892152 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665b068-c90b-4743-a4b1-26e1711bec95 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:51.897915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 331/666] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:51.899625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153ea2-8ec9-4930-9411-cfc6cb119021 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:51.900125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153ea2-8ec9-4930-9411-cfc6cb119021 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:51.900578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153ea2-8ec9-4930-9411-cfc6cb119021 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:51.906345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 332/667] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:51.907641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 336/668] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/role_assignments?user.id=cc21faf2237d405bba03e5309d93629f&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:51.907875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0153ea2-8ec9-4930-9411-cfc6cb119021 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.908614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665b068-c90b-4743-a4b1-26e1711bec95 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:51.924922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:52.023297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665b068-c90b-4743-a4b1-26e1711bec95 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.029195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0153ea2-8ec9-4930-9411-cfc6cb119021 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.062161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd4580-a468-4bb7-84d6-39ad7edb4411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.063247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd4580-a468-4bb7-84d6-39ad7edb4411 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.063780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd4580-a468-4bb7-84d6-39ad7edb4411 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.064397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd4580-a468-4bb7-84d6-39ad7edb4411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.068450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f16b8a-caa1-4b8d-8b9c-3ff9c9a70f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.069127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f16b8a-caa1-4b8d-8b9c-3ff9c9a70f31 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.069420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f16b8a-caa1-4b8d-8b9c-3ff9c9a70f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.069703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f16b8a-caa1-4b8d-8b9c-3ff9c9a70f31 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.070430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f16b8a-caa1-4b8d-8b9c-3ff9c9a70f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:52.070743 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f16b8a-caa1-4b8d-8b9c-3ff9c9a70f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:52.073538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f16b8a-caa1-4b8d-8b9c-3ff9c9a70f31 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:52.077285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd4580-a468-4bb7-84d6-39ad7edb4411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:52.078727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd4580-a468-4bb7-84d6-39ad7edb4411 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:52.088379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 333/669] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:52.089725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b70f-aeb5-4bbc-a426-6dddc13ffd8d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.090022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b70f-aeb5-4bbc-a426-6dddc13ffd8d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.090306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b70f-aeb5-4bbc-a426-6dddc13ffd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7bbd4580-a468-4bb7-84d6-39ad7edb4411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:52.102773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:52.103696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:52.104603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:52.104603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:52.104603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:52.104603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:52.106731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 337/670] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:52.107730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd4580-a468-4bb7-84d6-39ad7edb4411 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:52.112711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1146b70f-aeb5-4bbc-a426-6dddc13ffd8d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:52.113939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1146b70f-aeb5-4bbc-a426-6dddc13ffd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:52.164893 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:52.194554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:52.201403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 334/671] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:52.202434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146b70f-aeb5-4bbc-a426-6dddc13ffd8d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:52.234285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd4580-a468-4bb7-84d6-39ad7edb4411 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.280620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac57eb-12ed-4c31-bde8-da9796294a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.281984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac57eb-12ed-4c31-bde8-da9796294a2d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.282501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac57eb-12ed-4c31-bde8-da9796294a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.282955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac57eb-12ed-4c31-bde8-da9796294a2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.293209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac57eb-12ed-4c31-bde8-da9796294a2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:52.295540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac57eb-12ed-4c31-bde8-da9796294a2d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0eac57eb-12ed-4c31-bde8-da9796294a2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:52.315272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:52.316264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:52.317228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:52.317228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:52.317228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:52.317228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:52.317228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:52.317228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:52.317228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:52.317228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:52.320253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 338/672] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:52.325955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac57eb-12ed-4c31-bde8-da9796294a2d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:52.333458 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146b70f-aeb5-4bbc-a426-6dddc13ffd8d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.370663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313402b3-35b2-4bf7-950a-0f2344e5a71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.372199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313402b3-35b2-4bf7-950a-0f2344e5a71c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.372707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313402b3-35b2-4bf7-950a-0f2344e5a71c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.373196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313402b3-35b2-4bf7-950a-0f2344e5a71c None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.374185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313402b3-35b2-4bf7-950a-0f2344e5a71c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:52.374798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313402b3-35b2-4bf7-950a-0f2344e5a71c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:52.376489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313402b3-35b2-4bf7-950a-0f2344e5a71c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:52.389827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 335/673] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:52.390949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313402b3-35b2-4bf7-950a-0f2344e5a71c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:52.439850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:52.442968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac57eb-12ed-4c31-bde8-da9796294a2d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.461475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:52.479778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c8a17-72cd-4d66-ab33-ffdc8699dea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.480722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8a17-72cd-4d66-ab33-ffdc8699dea5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.481005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8a17-72cd-4d66-ab33-ffdc8699dea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.481277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8a17-72cd-4d66-ab33-ffdc8699dea5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.492088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c8a17-72cd-4d66-ab33-ffdc8699dea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e28c8a17-72cd-4d66-ab33-ffdc8699dea5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:52.500061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:52.501021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:52.501850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:52.506379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 339/674] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:52.507232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c8a17-72cd-4d66-ab33-ffdc8699dea5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:52.513354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313402b3-35b2-4bf7-950a-0f2344e5a71c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.549558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929027b-cbd3-4d63-9cad-33140d99086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.550762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929027b-cbd3-4d63-9cad-33140d99086f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.551276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929027b-cbd3-4d63-9cad-33140d99086f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.551791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929027b-cbd3-4d63-9cad-33140d99086f None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.552880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929027b-cbd3-4d63-9cad-33140d99086f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:52.553391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929027b-cbd3-4d63-9cad-33140d99086f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:52.555099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929027b-cbd3-4d63-9cad-33140d99086f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:52.568702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 336/675] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:52.570022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929027b-cbd3-4d63-9cad-33140d99086f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:52.612768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c8a17-72cd-4d66-ab33-ffdc8699dea5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.647838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd8087-13e7-4cb9-b7b5-66aee6897f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.648658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd8087-13e7-4cb9-b7b5-66aee6897f2f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.648926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd8087-13e7-4cb9-b7b5-66aee6897f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.649202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd8087-13e7-4cb9-b7b5-66aee6897f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.660480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd8087-13e7-4cb9-b7b5-66aee6897f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:52.661940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd8087-13e7-4cb9-b7b5-66aee6897f2f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e0bd8087-13e7-4cb9-b7b5-66aee6897f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:52.677250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:52.678162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:52.679064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:52.679064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:52.679064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:52.679064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:52.679064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:52.679064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:52.679064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:52.679064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:52.680748 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 340/676] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:52.681523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd8087-13e7-4cb9-b7b5-66aee6897f2f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:52.683947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929027b-cbd3-4d63-9cad-33140d99086f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.706363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:52.709847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:52.724059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead42446-95cf-42bf-b12b-2470ff207ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.724852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42446-95cf-42bf-b12b-2470ff207ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.725858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42446-95cf-42bf-b12b-2470ff207ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.725858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42446-95cf-42bf-b12b-2470ff207ea6 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.726154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead42446-95cf-42bf-b12b-2470ff207ea6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:52.726469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead42446-95cf-42bf-b12b-2470ff207ea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:52.728066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead42446-95cf-42bf-b12b-2470ff207ea6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:52.742726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 337/677] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:52.742726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df140b1a-0043-4297-8b9c-431b2bcf025e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.742726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df140b1a-0043-4297-8b9c-431b2bcf025e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.742726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df140b1a-0043-4297-8b9c-431b2bcf025e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.742726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 338/678] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:52.742726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df140b1a-0043-4297-8b9c-431b2bcf025e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:52.812780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd8087-13e7-4cb9-b7b5-66aee6897f2f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.848392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47532a8e-aa01-4c7a-85e6-680671d0c914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.849048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47532a8e-aa01-4c7a-85e6-680671d0c914 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.849332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47532a8e-aa01-4c7a-85e6-680671d0c914 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.849609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47532a8e-aa01-4c7a-85e6-680671d0c914 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.850439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47532a8e-aa01-4c7a-85e6-680671d0c914 None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=9fc23c141eff42f98d31deb2d8aaaca6) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:52.850755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47532a8e-aa01-4c7a-85e6-680671d0c914 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:52.852221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47532a8e-aa01-4c7a-85e6-680671d0c914 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:52.858209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 341/679] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:52 2022] GET /identity/v3/role_assignments?user.id=9fc23c141eff42f98d31deb2d8aaaca6&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:52.858323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df140b1a-0043-4297-8b9c-431b2bcf025e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:52.858818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47532a8e-aa01-4c7a-85e6-680671d0c914 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:52.913454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483080dd-daed-42aa-8233-d466a39890c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:52.915089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483080dd-daed-42aa-8233-d466a39890c5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.915654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483080dd-daed-42aa-8233-d466a39890c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.916130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483080dd-daed-42aa-8233-d466a39890c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.917860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483080dd-daed-42aa-8233-d466a39890c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:52.918415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483080dd-daed-42aa-8233-d466a39890c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:52.920938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483080dd-daed-42aa-8233-d466a39890c5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:52.939207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 339/680] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:52.940983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727a747-1ceb-42be-ac13-7261fe0ffd62 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:52.941502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727a747-1ceb-42be-ac13-7261fe0ffd62 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:52.942001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727a747-1ceb-42be-ac13-7261fe0ffd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:52.964260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d727a747-1ceb-42be-ac13-7261fe0ffd62 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:52.966705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d727a747-1ceb-42be-ac13-7261fe0ffd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:52.969331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:52.989061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47532a8e-aa01-4c7a-85e6-680671d0c914 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:53.017095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:53.027829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530795a8-1497-4560-882f-5ca155be228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:53.028644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530795a8-1497-4560-882f-5ca155be228a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:53.028928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530795a8-1497-4560-882f-5ca155be228a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:53.029251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530795a8-1497-4560-882f-5ca155be228a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:53.042278 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530795a8-1497-4560-882f-5ca155be228a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:53.043811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530795a8-1497-4560-882f-5ca155be228a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:53.058354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 340/681] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:53.060162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727a747-1ceb-42be-ac13-7261fe0ffd62 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-530795a8-1497-4560-882f-5ca155be228a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:53.081189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:53.082059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:53.082918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:53.082918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:53.082918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:53.082918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:53.082918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 342/682] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:53.089304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530795a8-1497-4560-882f-5ca155be228a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:53.175448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727a747-1ceb-42be-ac13-7261fe0ffd62 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:53.212562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef12666-8874-4913-ad0a-b02d902b1437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:53.213603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef12666-8874-4913-ad0a-b02d902b1437 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:53.214024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef12666-8874-4913-ad0a-b02d902b1437 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:53.214597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef12666-8874-4913-ad0a-b02d902b1437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:53.216645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef12666-8874-4913-ad0a-b02d902b1437 None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=c57d44c19d15496e96c6e37510ce67fc) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:53.217405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef12666-8874-4913-ad0a-b02d902b1437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:53.219805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530795a8-1497-4560-882f-5ca155be228a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:53.220643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef12666-8874-4913-ad0a-b02d902b1437 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:53.229094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 341/683] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:53 2022] GET /identity/v3/role_assignments?user.id=c57d44c19d15496e96c6e37510ce67fc&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:53.230363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef12666-8874-4913-ad0a-b02d902b1437 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:53.292078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb7a1a-57be-423c-9c29-d0029c22db94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:53.294537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb7a1a-57be-423c-9c29-d0029c22db94 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:53.295796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb7a1a-57be-423c-9c29-d0029c22db94 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:53.296257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb7a1a-57be-423c-9c29-d0029c22db94 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:53.297206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb7a1a-57be-423c-9c29-d0029c22db94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:53.297932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb7a1a-57be-423c-9c29-d0029c22db94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:53.299264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:53.301413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb7a1a-57be-423c-9c29-d0029c22db94 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:53.309589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:53.326050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 343/684] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:53.332880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb7a1a-57be-423c-9c29-d0029c22db94 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:53.456770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef12666-8874-4913-ad0a-b02d902b1437 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:53.462801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb7a1a-57be-423c-9c29-d0029c22db94 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:53.512809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d05cf2-5124-4359-af81-29619fb6b5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:53.513541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d05cf2-5124-4359-af81-29619fb6b5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:53.513827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d05cf2-5124-4359-af81-29619fb6b5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:53.514107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d05cf2-5124-4359-af81-29619fb6b5a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:53.518434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf747ea-2118-4c33-92ab-c7a4dc7d6f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:53.519239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf747ea-2118-4c33-92ab-c7a4dc7d6f6f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:53.519542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf747ea-2118-4c33-92ab-c7a4dc7d6f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:53.519815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf747ea-2118-4c33-92ab-c7a4dc7d6f6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:53.527409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf747ea-2118-4c33-92ab-c7a4dc7d6f6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:53.528922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf747ea-2118-4c33-92ab-c7a4dc7d6f6f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:53.529044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d05cf2-5124-4359-af81-29619fb6b5a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-04d05cf2-5124-4359-af81-29619fb6b5a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:53.539599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:53.540551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:53.541372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 342/685] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-4cf747ea-2118-4c33-92ab-c7a4dc7d6f6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:53.544167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:53.545028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:28:53.546222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:53.549350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d05cf2-5124-4359-af81-29619fb6b5a4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:53.549961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 344/686] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:53.551073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf747ea-2118-4c33-92ab-c7a4dc7d6f6f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:53.615904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:53.671821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf747ea-2118-4c33-92ab-c7a4dc7d6f6f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:53.713292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe838-1a3f-4238-9b7b-df1e5d3ccb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:53.715182 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe838-1a3f-4238-9b7b-df1e5d3ccb17 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:53.715819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:53.718005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe838-1a3f-4238-9b7b-df1e5d3ccb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:53.718791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe838-1a3f-4238-9b7b-df1e5d3ccb17 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:53.721185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fe838-1a3f-4238-9b7b-df1e5d3ccb17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:53.722942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fe838-1a3f-4238-9b7b-df1e5d3ccb17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:53.724684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fe838-1a3f-4238-9b7b-df1e5d3ccb17 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:53.728047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d05cf2-5124-4359-af81-29619fb6b5a4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:53.770105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 345/687] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 13 20:28:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:53.775509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe838-1a3f-4238-9b7b-df1e5d3ccb17 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:53.784672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825c709-a63a-4f04-89e8-b6532b64e622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:53.785356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825c709-a63a-4f04-89e8-b6532b64e622 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:53.785640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825c709-a63a-4f04-89e8-b6532b64e622 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:53.785918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825c709-a63a-4f04-89e8-b6532b64e622 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:53.786667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825c709-a63a-4f04-89e8-b6532b64e622 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:53.786982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825c709-a63a-4f04-89e8-b6532b64e622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:53.788403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825c709-a63a-4f04-89e8-b6532b64e622 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:53.801427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 343/688] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:53.808648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825c709-a63a-4f04-89e8-b6532b64e622 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:53.945348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe838-1a3f-4238-9b7b-df1e5d3ccb17 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:53.964885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:53.985107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825c709-a63a-4f04-89e8-b6532b64e622 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:53.997389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1544-1d3d-4955-bd21-1cd74f1838ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:53.998051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1544-1d3d-4955-bd21-1cd74f1838ce None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:53.998333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1544-1d3d-4955-bd21-1cd74f1838ce None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:53.998607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1544-1d3d-4955-bd21-1cd74f1838ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.049107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a3a2e-3d9f-49b2-b99d-4a1191e9a6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:54.049842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a3a2e-3d9f-49b2-b99d-4a1191e9a6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:54.050127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a3a2e-3d9f-49b2-b99d-4a1191e9a6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:54.050403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a3a2e-3d9f-49b2-b99d-4a1191e9a6a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.075560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a3a2e-3d9f-49b2-b99d-4a1191e9a6a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:54.077783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a3a2e-3d9f-49b2-b99d-4a1191e9a6a1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2f5a3a2e-3d9f-49b2-b99d-4a1191e9a6a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:54.103532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:54.104490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:54.105448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:54.105448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:54.105448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:54.105448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:54.105448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 344/689] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:54.111786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a3a2e-3d9f-49b2-b99d-4a1191e9a6a1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:54.129702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:54.142664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec1544-1d3d-4955-bd21-1cd74f1838ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:54.145663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec1544-1d3d-4955-bd21-1cd74f1838ce None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:54.270832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a3a2e-3d9f-49b2-b99d-4a1191e9a6a1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:54.286485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 346/690] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:54.288136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327eecd-2f47-4ddd-8476-37674631fb98 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:54.288630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327eecd-2f47-4ddd-8476-37674631fb98 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:54.289123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327eecd-2f47-4ddd-8476-37674631fb98 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.290804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 347/691] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:54.292674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327eecd-2f47-4ddd-8476-37674631fb98 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:54.314493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:54.319884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e5215-91e1-4603-926a-6e7d7e29a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:54.320813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e5215-91e1-4603-926a-6e7d7e29a53f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:54.321297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e5215-91e1-4603-926a-6e7d7e29a53f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:54.321764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e5215-91e1-4603-926a-6e7d7e29a53f None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.322687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e5215-91e1-4603-926a-6e7d7e29a53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:54.323551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e5215-91e1-4603-926a-6e7d7e29a53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:54.325179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e5215-91e1-4603-926a-6e7d7e29a53f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:54.338147 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 345/692] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:54.343872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e81d62-3fd0-4c2e-9dc5-9e693becc76d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:54.344435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e81d62-3fd0-4c2e-9dc5-9e693becc76d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:54.344930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e81d62-3fd0-4c2e-9dc5-9e693becc76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.361709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-10e81d62-3fd0-4c2e-9dc5-9e693becc76d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:54.363166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-10e81d62-3fd0-4c2e-9dc5-9e693becc76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:54.428601 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:54.429643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327eecd-2f47-4ddd-8476-37674631fb98 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:54.434003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 346/693] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:54.435170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e81d62-3fd0-4c2e-9dc5-9e693becc76d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:54.484706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46d67c-5697-47ef-9ccd-3653a4733b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:54.485685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46d67c-5697-47ef-9ccd-3653a4733b23 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:54.486063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46d67c-5697-47ef-9ccd-3653a4733b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:54.486440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46d67c-5697-47ef-9ccd-3653a4733b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.498764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46d67c-5697-47ef-9ccd-3653a4733b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:54.500504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46d67c-5697-47ef-9ccd-3653a4733b23 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3a46d67c-5697-47ef-9ccd-3653a4733b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:54.519340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:54.520461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:54.521555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:54.521555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:54.521555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:54.521555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:54.523097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 348/694] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:54.528091 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46d67c-5697-47ef-9ccd-3653a4733b23 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:54.578530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e81d62-3fd0-4c2e-9dc5-9e693becc76d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:54.597293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:54.632956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b46ed0-6e9a-4cdf-86ad-1ad4a7d0eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:54.634453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b46ed0-6e9a-4cdf-86ad-1ad4a7d0eccc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:54.635236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b46ed0-6e9a-4cdf-86ad-1ad4a7d0eccc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:54.636371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b46ed0-6e9a-4cdf-86ad-1ad4a7d0eccc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.638891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46d67c-5697-47ef-9ccd-3653a4733b23 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:54.647210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b46ed0-6e9a-4cdf-86ad-1ad4a7d0eccc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:54.649461 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b46ed0-6e9a-4cdf-86ad-1ad4a7d0eccc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e0b46ed0-6e9a-4cdf-86ad-1ad4a7d0eccc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:54.670833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:54.672267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:28:54.673532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:54.677445 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 347/695] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 13 20:28:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:54.678531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b46ed0-6e9a-4cdf-86ad-1ad4a7d0eccc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:54.680658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c0bb7-866c-4657-9f04-b84db98df451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:54.681709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c0bb7-866c-4657-9f04-b84db98df451 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:54.682107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c0bb7-866c-4657-9f04-b84db98df451 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:54.682536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c0bb7-866c-4657-9f04-b84db98df451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.695567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c0bb7-866c-4657-9f04-b84db98df451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:54.697618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c0bb7-866c-4657-9f04-b84db98df451 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c15c0bb7-866c-4657-9f04-b84db98df451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:54.719223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:54.720396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:54.721481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:54.721481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:54.721481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:54.721481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:54.727557 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 349/696] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:54.727557 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c0bb7-866c-4657-9f04-b84db98df451 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:54.765590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:54.815978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b46ed0-6e9a-4cdf-86ad-1ad4a7d0eccc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:54.852340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060a6de-4c3c-4edc-a7f3-b51fcf054e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:54.852936 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060a6de-4c3c-4edc-a7f3-b51fcf054e51 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:54.853223 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060a6de-4c3c-4edc-a7f3-b51fcf054e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:54.853500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060a6de-4c3c-4edc-a7f3-b51fcf054e51 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.854221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060a6de-4c3c-4edc-a7f3-b51fcf054e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:54.854528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060a6de-4c3c-4edc-a7f3-b51fcf054e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:54.855624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c0bb7-866c-4657-9f04-b84db98df451 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:54.856028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060a6de-4c3c-4edc-a7f3-b51fcf054e51 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:54.868933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 348/697] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:54.869846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060a6de-4c3c-4edc-a7f3-b51fcf054e51 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:54.881455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:54.908479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e741ce-b67f-433f-ac90-bb2e67923c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:54.909553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e741ce-b67f-433f-ac90-bb2e67923c59 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:54.910034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e741ce-b67f-433f-ac90-bb2e67923c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:54.910593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e741ce-b67f-433f-ac90-bb2e67923c59 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:54.911664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e741ce-b67f-433f-ac90-bb2e67923c59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:54.912689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e741ce-b67f-433f-ac90-bb2e67923c59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:54.914892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e741ce-b67f-433f-ac90-bb2e67923c59 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:54.944102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 350/698] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 13 20:28:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:54.945590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e741ce-b67f-433f-ac90-bb2e67923c59 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:55.001882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060a6de-4c3c-4edc-a7f3-b51fcf054e51 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:55.028994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:55.048307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28e604-7ec6-46d5-bf41-cb4919a3400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:55.049224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28e604-7ec6-46d5-bf41-cb4919a3400f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.049540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28e604-7ec6-46d5-bf41-cb4919a3400f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.050181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28e604-7ec6-46d5-bf41-cb4919a3400f None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.051177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28e604-7ec6-46d5-bf41-cb4919a3400f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:55.051511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28e604-7ec6-46d5-bf41-cb4919a3400f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:55.053856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28e604-7ec6-46d5-bf41-cb4919a3400f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:55.056099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e741ce-b67f-433f-ac90-bb2e67923c59 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:55.069029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 349/699] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:55.070025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28e604-7ec6-46d5-bf41-cb4919a3400f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:55.099919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8f23c-c117-4c7a-b7d0-d1a0119a0e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:55.100731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8f23c-c117-4c7a-b7d0-d1a0119a0e9c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.101013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8f23c-c117-4c7a-b7d0-d1a0119a0e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.101322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8f23c-c117-4c7a-b7d0-d1a0119a0e9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.108618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8f23c-c117-4c7a-b7d0-d1a0119a0e9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:55.110012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8f23c-c117-4c7a-b7d0-d1a0119a0e9c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ded8f23c-c117-4c7a-b7d0-d1a0119a0e9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:55.125999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:55.126936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:55.127896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:55.127896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:55.127896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:55.127896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:55.127896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:55.127896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:55.127896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:55.127896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:55.127896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 351/700] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:55.128377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8f23c-c117-4c7a-b7d0-d1a0119a0e9c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:55.165567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:55.191146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28e604-7ec6-46d5-bf41-cb4919a3400f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:55.230814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7aa786-a6c7-4282-87c8-8e6682683da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:55.231632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7aa786-a6c7-4282-87c8-8e6682683da8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.231933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7aa786-a6c7-4282-87c8-8e6682683da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.232212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7aa786-a6c7-4282-87c8-8e6682683da8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.238016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8f23c-c117-4c7a-b7d0-d1a0119a0e9c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:55.274767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3422e3-6654-453b-b0c6-b1c91fd5224d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:55.275443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3422e3-6654-453b-b0c6-b1c91fd5224d None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.275767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3422e3-6654-453b-b0c6-b1c91fd5224d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.276043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3422e3-6654-453b-b0c6-b1c91fd5224d None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.276838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3422e3-6654-453b-b0c6-b1c91fd5224d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:55.278283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3422e3-6654-453b-b0c6-b1c91fd5224d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:55.286568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:55.302232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 352/701] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:28:55 2022] POST /identity/v3/users => generated 254 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:28:55.303907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d83b7-9677-4171-8d4c-6be32c49da26 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.304195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d83b7-9677-4171-8d4c-6be32c49da26 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.304476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d83b7-9677-4171-8d4c-6be32c49da26 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.306126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 353/702] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:55.311076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d83b7-9677-4171-8d4c-6be32c49da26 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:55.318404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7aa786-a6c7-4282-87c8-8e6682683da8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:55.320358 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7aa786-a6c7-4282-87c8-8e6682683da8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:55.434022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:55.448031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d83b7-9677-4171-8d4c-6be32c49da26 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:55.461954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 350/703] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:55.464788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121c009-b711-47ba-b14e-3b5e923b808b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.465330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121c009-b711-47ba-b14e-3b5e923b808b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.466094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121c009-b711-47ba-b14e-3b5e923b808b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.485910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a121c009-b711-47ba-b14e-3b5e923b808b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:55.487539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a121c009-b711-47ba-b14e-3b5e923b808b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:55.492557 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecd4a6-b9a7-4a5c-9376-8a6a91348489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:55.493238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecd4a6-b9a7-4a5c-9376-8a6a91348489 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.493523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecd4a6-b9a7-4a5c-9376-8a6a91348489 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.493797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecd4a6-b9a7-4a5c-9376-8a6a91348489 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.494588 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbecd4a6-b9a7-4a5c-9376-8a6a91348489 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:55.494896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbecd4a6-b9a7-4a5c-9376-8a6a91348489 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:55.496404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbecd4a6-b9a7-4a5c-9376-8a6a91348489 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:55.510858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 354/704] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:55.511803 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecd4a6-b9a7-4a5c-9376-8a6a91348489 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:55.579384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 351/705] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:55.580613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121c009-b711-47ba-b14e-3b5e923b808b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:55.593030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:55.646159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecd4a6-b9a7-4a5c-9376-8a6a91348489 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:55.683025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ea9aa-0504-4ace-94c4-0980fbd167ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:55.683855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ea9aa-0504-4ace-94c4-0980fbd167ee None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.684202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ea9aa-0504-4ace-94c4-0980fbd167ee None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.684408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ea9aa-0504-4ace-94c4-0980fbd167ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.691042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121c009-b711-47ba-b14e-3b5e923b808b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:55.695789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ea9aa-0504-4ace-94c4-0980fbd167ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:55.697419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ea9aa-0504-4ace-94c4-0980fbd167ee None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-202ea9aa-0504-4ace-94c4-0980fbd167ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:55.716425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:55.717402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:55.718300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:55.718300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:55.718300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:55.718300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:55.719952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 355/706] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:55.720861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31926fb-8a52-4d47-9bbd-7fcc452b35a3 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.721148 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31926fb-8a52-4d47-9bbd-7fcc452b35a3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.721427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31926fb-8a52-4d47-9bbd-7fcc452b35a3 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.723478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 356/707] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:55.724230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31926fb-8a52-4d47-9bbd-7fcc452b35a3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:55.728437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:55.732777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f6705-b3bb-4207-9f26-c59dc39c5bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:55.733445 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f6705-b3bb-4207-9f26-c59dc39c5bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.733729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f6705-b3bb-4207-9f26-c59dc39c5bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.734006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f6705-b3bb-4207-9f26-c59dc39c5bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.734874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f6705-b3bb-4207-9f26-c59dc39c5bf2 None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=3f27793f58c74ffb9a7a8e4cdddb8169) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:55.735146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f6705-b3bb-4207-9f26-c59dc39c5bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:55.736629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f6705-b3bb-4207-9f26-c59dc39c5bf2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:55.747494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 352/708] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:55 2022] GET /identity/v3/role_assignments?user.id=3f27793f58c74ffb9a7a8e4cdddb8169&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:55.747494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f6705-b3bb-4207-9f26-c59dc39c5bf2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:55.843085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31926fb-8a52-4d47-9bbd-7fcc452b35a3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:55.850769 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:55.882256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16b32d-0bc4-425d-aec7-1a74dd24be29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:55.883091 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16b32d-0bc4-425d-aec7-1a74dd24be29 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.883379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16b32d-0bc4-425d-aec7-1a74dd24be29 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.883691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16b32d-0bc4-425d-aec7-1a74dd24be29 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.884052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f6705-b3bb-4207-9f26-c59dc39c5bf2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:55.884357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16b32d-0bc4-425d-aec7-1a74dd24be29 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:55.885866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16b32d-0bc4-425d-aec7-1a74dd24be29 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3b16b32d-0bc4-425d-aec7-1a74dd24be29 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:55.905218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:55.906235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:28:55.907070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:55.907870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 357/709] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Jul 13 20:28:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:55.911484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb6642-2ffd-40c8-9ec9-01b9c4c22828 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.911484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb6642-2ffd-40c8-9ec9-01b9c4c22828 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.911484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb6642-2ffd-40c8-9ec9-01b9c4c22828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.929845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a0eb6642-2ffd-40c8-9ec9-01b9c4c22828 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:55.931037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a0eb6642-2ffd-40c8-9ec9-01b9c4c22828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:55.934678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6eeb65-f323-4821-8ef3-4e4e161bff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:55.935389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6eeb65-f323-4821-8ef3-4e4e161bff70 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:55.935715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6eeb65-f323-4821-8ef3-4e4e161bff70 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:55.935995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6eeb65-f323-4821-8ef3-4e4e161bff70 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:55.936720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6eeb65-f323-4821-8ef3-4e4e161bff70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:55.937725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6eeb65-f323-4821-8ef3-4e4e161bff70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:55.940476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6eeb65-f323-4821-8ef3-4e4e161bff70 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:55.957175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 353/710] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:55.957175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6eeb65-f323-4821-8ef3-4e4e161bff70 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:56.014480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:56.020798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 358/711] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:56.022095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb6642-2ffd-40c8-9ec9-01b9c4c22828 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:56.070655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6eeb65-f323-4821-8ef3-4e4e161bff70 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:56.107263 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1687be-e535-44c9-beb5-4c725db9a1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:56.107969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1687be-e535-44c9-beb5-4c725db9a1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:56.108254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1687be-e535-44c9-beb5-4c725db9a1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:56.108534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1687be-e535-44c9-beb5-4c725db9a1a5 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:56.109242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1687be-e535-44c9-beb5-4c725db9a1a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:56.109695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1687be-e535-44c9-beb5-4c725db9a1a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:56.111693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1687be-e535-44c9-beb5-4c725db9a1a5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:56.113367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:56.118718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 354/712] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 13 20:28:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:56.119545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1687be-e535-44c9-beb5-4c725db9a1a5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:56.152834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb6642-2ffd-40c8-9ec9-01b9c4c22828 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:56.201604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e754c4-7ef0-4879-9b4c-65a317de0082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:56.203395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e754c4-7ef0-4879-9b4c-65a317de0082 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:56.204115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e754c4-7ef0-4879-9b4c-65a317de0082 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:56.204581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e754c4-7ef0-4879-9b4c-65a317de0082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:56.214216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e754c4-7ef0-4879-9b4c-65a317de0082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:56.216281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e754c4-7ef0-4879-9b4c-65a317de0082 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-27e754c4-7ef0-4879-9b4c-65a317de0082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:28:56.235698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:56.236685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:56.237654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:28:56.237654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:28:56.237654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:28:56.237654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:28:56.237654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:28:56.237654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:28:56.237654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:28:56.237654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:56.238070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1687be-e535-44c9-beb5-4c725db9a1a5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:56.241005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 359/713] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:56.241995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e754c4-7ef0-4879-9b4c-65a317de0082 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:56.282360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac05ae-706a-4c65-9434-9eeae7f5ee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:56.283321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac05ae-706a-4c65-9434-9eeae7f5ee58 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:56.283725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac05ae-706a-4c65-9434-9eeae7f5ee58 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:56.284186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac05ae-706a-4c65-9434-9eeae7f5ee58 None admin] PATH_INFO: `/v3/services/network` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:56.284843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac05ae-706a-4c65-9434-9eeae7f5ee58 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:56.286404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac05ae-706a-4c65-9434-9eeae7f5ee58 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1fac05ae-706a-4c65-9434-9eeae7f5ee58 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:56.300770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:56.301942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:28:56.302945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:56.305115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 355/714] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:28:56 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:56.306068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac05ae-706a-4c65-9434-9eeae7f5ee58 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:56.319955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:56.369001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e754c4-7ef0-4879-9b4c-65a317de0082 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:56.372080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:56.412261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acf524-8e2f-4e67-afe0-1a94c82815c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:56.413627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acf524-8e2f-4e67-afe0-1a94c82815c3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:56.414064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acf524-8e2f-4e67-afe0-1a94c82815c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:56.414715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acf524-8e2f-4e67-afe0-1a94c82815c3 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:56.415616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acf524-8e2f-4e67-afe0-1a94c82815c3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:56.416066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acf524-8e2f-4e67-afe0-1a94c82815c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:56.416616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac05ae-706a-4c65-9434-9eeae7f5ee58 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:56.417863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acf524-8e2f-4e67-afe0-1a94c82815c3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:56.425643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 360/715] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 13 20:28:56 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:56.427013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acf524-8e2f-4e67-afe0-1a94c82815c3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:56.463201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07234164-eee3-4e35-beb2-98563c28e365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:56.465701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07234164-eee3-4e35-beb2-98563c28e365 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:56.466429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07234164-eee3-4e35-beb2-98563c28e365 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:56.468328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07234164-eee3-4e35-beb2-98563c28e365 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:56.470256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07234164-eee3-4e35-beb2-98563c28e365 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:56.471493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07234164-eee3-4e35-beb2-98563c28e365 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:56.474888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07234164-eee3-4e35-beb2-98563c28e365 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:56.504084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 356/716] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:56.509921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07234164-eee3-4e35-beb2-98563c28e365 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:56.618738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acf524-8e2f-4e67-afe0-1a94c82815c3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:56.649461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:56.657480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07234164-eee3-4e35-beb2-98563c28e365 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:56.663788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf76918-6296-4316-b460-236a86939c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:56.664718 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf76918-6296-4316-b460-236a86939c28 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:56.665150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf76918-6296-4316-b460-236a86939c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:56.665571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf76918-6296-4316-b460-236a86939c28 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:56.666548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf76918-6296-4316-b460-236a86939c28 None admin] RBAC: Adding query filter params (name=network) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:56.666998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf76918-6296-4316-b460-236a86939c28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:56.668775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf76918-6296-4316-b460-236a86939c28 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:56.674543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 361/717] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:28:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:56.681349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf76918-6296-4316-b460-236a86939c28 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:56.683660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:56.699133 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9930c-b67e-45af-86f5-d0dd0a6dbf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:56.699818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9930c-b67e-45af-86f5-d0dd0a6dbf46 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:56.700103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9930c-b67e-45af-86f5-d0dd0a6dbf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:56.700381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9930c-b67e-45af-86f5-d0dd0a6dbf46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:56.701369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9930c-b67e-45af-86f5-d0dd0a6dbf46 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=e80bb5b10ce945cabbfb88ca5df92e3c) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:56.701511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9930c-b67e-45af-86f5-d0dd0a6dbf46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:56.702964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9930c-b67e-45af-86f5-d0dd0a6dbf46 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:56.709220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 357/718] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:28:56 2022] GET /identity/v3/role_assignments?user.id=e80bb5b10ce945cabbfb88ca5df92e3c&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:56.798263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf76918-6296-4316-b460-236a86939c28 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:56.834921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb875c-62d9-4931-bcb1-1a88f091cfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:56.835897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb875c-62d9-4931-bcb1-1a88f091cfda None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:56.836330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb875c-62d9-4931-bcb1-1a88f091cfda None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:56.836776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb875c-62d9-4931-bcb1-1a88f091cfda None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:56.837634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb875c-62d9-4931-bcb1-1a88f091cfda None admin] RBAC: Adding query filter params (type=network) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:56.838108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb875c-62d9-4931-bcb1-1a88f091cfda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:56.839734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb875c-62d9-4931-bcb1-1a88f091cfda None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:56.845178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 362/719] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:28:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:57.642602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5d2d4-f4bd-4eba-b670-2e40515926ca None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:57.643125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5d2d4-f4bd-4eba-b670-2e40515926ca None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:57.643125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5d2d4-f4bd-4eba-b670-2e40515926ca None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:57.644777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 358/720] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:57.649962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac99b49-1d95-4439-a4a5-923744bfe882 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:57.650355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac99b49-1d95-4439-a4a5-923744bfe882 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:57.650537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac99b49-1d95-4439-a4a5-923744bfe882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:57.671935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-aac99b49-1d95-4439-a4a5-923744bfe882 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:57.673122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aac99b49-1d95-4439-a4a5-923744bfe882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:57.738042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:57.763045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 363/721] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:57.801829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5d2d4-f4bd-4eba-b670-2e40515926ca None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:57.936457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5d2d4-f4bd-4eba-b670-2e40515926ca None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:57.986103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500b436-b72c-43a6-8786-72361b0dda20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:57.986904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500b436-b72c-43a6-8786-72361b0dda20 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:57.987184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500b436-b72c-43a6-8786-72361b0dda20 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:57.987472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500b436-b72c-43a6-8786-72361b0dda20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:58.004439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500b436-b72c-43a6-8786-72361b0dda20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8500b436-b72c-43a6-8786-72361b0dda20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:58.012218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:58.013137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:58.013999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 359/722] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:28:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:58.019378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac99b49-1d95-4439-a4a5-923744bfe882 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:58.065012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdb526-e30b-4fe3-b837-7b47bc59bf02 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:58.065349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdb526-e30b-4fe3-b837-7b47bc59bf02 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:58.065570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdb526-e30b-4fe3-b837-7b47bc59bf02 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:58.067150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 360/723] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:28:58.087067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869828f-9a75-4fb6-b59a-7d5dda35b384 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:58.087404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869828f-9a75-4fb6-b59a-7d5dda35b384 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:58.087667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869828f-9a75-4fb6-b59a-7d5dda35b384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:58.128508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f869828f-9a75-4fb6-b59a-7d5dda35b384 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:58.129723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f869828f-9a75-4fb6-b59a-7d5dda35b384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:58.139929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac99b49-1d95-4439-a4a5-923744bfe882 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:58.179028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f367a7-08fe-4cdc-b2be-15f1502b79ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:58.179860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f367a7-08fe-4cdc-b2be-15f1502b79ba None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:58.180157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f367a7-08fe-4cdc-b2be-15f1502b79ba None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:58.180439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f367a7-08fe-4cdc-b2be-15f1502b79ba None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:58.181439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f367a7-08fe-4cdc-b2be-15f1502b79ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:58.181754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f367a7-08fe-4cdc-b2be-15f1502b79ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:58.183203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f367a7-08fe-4cdc-b2be-15f1502b79ba None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:58.192876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 364/724] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:28:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:58.197652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f367a7-08fe-4cdc-b2be-15f1502b79ba None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:58.243953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:58.249730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:58.311587 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 361/725] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:28:58.332952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869828f-9a75-4fb6-b59a-7d5dda35b384 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:58.377501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f367a7-08fe-4cdc-b2be-15f1502b79ba None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:58.444301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869828f-9a75-4fb6-b59a-7d5dda35b384 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:58.457510 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3f10-b1f8-4491-b495-f8c0a7e56565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:58.458637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3f10-b1f8-4491-b495-f8c0a7e56565 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:58.459066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3f10-b1f8-4491-b495-f8c0a7e56565 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:58.459496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3f10-b1f8-4491-b495-f8c0a7e56565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:58.482532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da372eb-9cfe-4059-b029-6ad47833007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:58.483758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da372eb-9cfe-4059-b029-6ad47833007d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:58.484283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da372eb-9cfe-4059-b029-6ad47833007d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:58.484660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da372eb-9cfe-4059-b029-6ad47833007d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:58.485222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a3f10-b1f8-4491-b495-f8c0a7e56565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:58.486853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a3f10-b1f8-4491-b495-f8c0a7e56565 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:58.504507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da372eb-9cfe-4059-b029-6ad47833007d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e10a3f10-b1f8-4491-b495-f8c0a7e56565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:58.522470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:58.523417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:58.524389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:58.524389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:58.524389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:58.524389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:58.524389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 365/726] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1da372eb-9cfe-4059-b029-6ad47833007d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:28:58.525844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:58.526924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:58.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 362/727] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:58.532836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3f10-b1f8-4491-b495-f8c0a7e56565 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:58.533137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da372eb-9cfe-4059-b029-6ad47833007d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:58.636574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:58.638743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:58.646217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3f10-b1f8-4491-b495-f8c0a7e56565 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:58.685740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc382727-3b98-4a70-bf5d-bc68bd117901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:58.687324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc382727-3b98-4a70-bf5d-bc68bd117901 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:58.687834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc382727-3b98-4a70-bf5d-bc68bd117901 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:58.688354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc382727-3b98-4a70-bf5d-bc68bd117901 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:58.689283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc382727-3b98-4a70-bf5d-bc68bd117901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:58.689784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc382727-3b98-4a70-bf5d-bc68bd117901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:58.691707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc382727-3b98-4a70-bf5d-bc68bd117901 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:58.707241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 366/728] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:58.713298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc382727-3b98-4a70-bf5d-bc68bd117901 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:58.769595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da372eb-9cfe-4059-b029-6ad47833007d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:58.817143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc382727-3b98-4a70-bf5d-bc68bd117901 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:58.851942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2753e-6014-44e4-abb7-8d62892c4ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:58.852151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758333f8-24ea-41ef-aa30-a9a59164e063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:58.852807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758333f8-24ea-41ef-aa30-a9a59164e063 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:58.853089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758333f8-24ea-41ef-aa30-a9a59164e063 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:58.853366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758333f8-24ea-41ef-aa30-a9a59164e063 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:58.853881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2753e-6014-44e4-abb7-8d62892c4ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:58.854117 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758333f8-24ea-41ef-aa30-a9a59164e063 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:58.854426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758333f8-24ea-41ef-aa30-a9a59164e063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:58.854958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2753e-6014-44e4-abb7-8d62892c4ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:58.855437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2753e-6014-44e4-abb7-8d62892c4ff6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:58.855978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758333f8-24ea-41ef-aa30-a9a59164e063 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:58.863912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2753e-6014-44e4-abb7-8d62892c4ff6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:58.865602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2753e-6014-44e4-abb7-8d62892c4ff6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:58.874843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 363/729] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:28:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:58.880230 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758333f8-24ea-41ef-aa30-a9a59164e063 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-43c2753e-6014-44e4-abb7-8d62892c4ff6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:58.882144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:58.883044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:28:58.884007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:58.887783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 367/730] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:28:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:58.893346 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2753e-6014-44e4-abb7-8d62892c4ff6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:58.910236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:59.039878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:59.044980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2753e-6014-44e4-abb7-8d62892c4ff6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:59.045375 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758333f8-24ea-41ef-aa30-a9a59164e063 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:59.107316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2b166-6a1e-4da4-9a4f-475756409a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:59.108028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2b166-6a1e-4da4-9a4f-475756409a63 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.108314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2b166-6a1e-4da4-9a4f-475756409a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.108593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2b166-6a1e-4da4-9a4f-475756409a63 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.109448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2b166-6a1e-4da4-9a4f-475756409a63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:59.109762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2b166-6a1e-4da4-9a4f-475756409a63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:59.111199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2b166-6a1e-4da4-9a4f-475756409a63 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:59.114103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901174d-5276-482a-b59d-4b30aca1b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:59.114918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901174d-5276-482a-b59d-4b30aca1b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.115200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901174d-5276-482a-b59d-4b30aca1b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.115490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901174d-5276-482a-b59d-4b30aca1b4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.133003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901174d-5276-482a-b59d-4b30aca1b4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:59.134482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901174d-5276-482a-b59d-4b30aca1b4d5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:59.151953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 368/731] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 13 20:28:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:28:59.153303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72453e21-3e52-4c5e-b615-cb7cb8b23a05 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.153592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72453e21-3e52-4c5e-b615-cb7cb8b23a05 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.153871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72453e21-3e52-4c5e-b615-cb7cb8b23a05 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b901174d-5276-482a-b59d-4b30aca1b4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:59.163408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:59.164350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:59.165215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:59.165215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:59.165215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:59.165215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:28:59.165445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 369/732] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:59.165946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72453e21-3e52-4c5e-b615-cb7cb8b23a05 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:59.166366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 364/733] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:28:59.167637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e950c-70df-4b2a-a499-d4b883a1a338 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.167927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e950c-70df-4b2a-a499-d4b883a1a338 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.168206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e950c-70df-4b2a-a499-d4b883a1a338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.185593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7b2e950c-70df-4b2a-a499-d4b883a1a338 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:59.186788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7b2e950c-70df-4b2a-a499-d4b883a1a338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:59.239754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:59.262690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 365/734] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:59.263766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e950c-70df-4b2a-a499-d4b883a1a338 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:59.279508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72453e21-3e52-4c5e-b615-cb7cb8b23a05 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:59.326969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b179cb-b294-4b1f-94fb-771bb5cdd3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:59.327914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b179cb-b294-4b1f-94fb-771bb5cdd3bf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.328198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b179cb-b294-4b1f-94fb-771bb5cdd3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.328565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b179cb-b294-4b1f-94fb-771bb5cdd3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.351313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:59.371944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e950c-70df-4b2a-a499-d4b883a1a338 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:59.401942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b179cb-b294-4b1f-94fb-771bb5cdd3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:59.403425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b179cb-b294-4b1f-94fb-771bb5cdd3bf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:59.408077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc10313-3027-43d9-abaa-4fadf78169d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:59.408968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc10313-3027-43d9-abaa-4fadf78169d3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.409422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc10313-3027-43d9-abaa-4fadf78169d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.409865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc10313-3027-43d9-abaa-4fadf78169d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.410749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc10313-3027-43d9-abaa-4fadf78169d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:59.411644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc10313-3027-43d9-abaa-4fadf78169d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:59.413528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc10313-3027-43d9-abaa-4fadf78169d3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:59.425613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 366/735] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:59.427089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3380f-9887-463e-9251-a83a0088fae1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.427378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3380f-9887-463e-9251-a83a0088fae1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.427690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3380f-9887-463e-9251-a83a0088fae1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.428906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 367/736] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:59.429570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3380f-9887-463e-9251-a83a0088fae1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:59.476784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:59.507828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 370/737] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:28:59.509153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf70a0-7079-4953-8a91-87fa369b8d11 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.509422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf70a0-7079-4953-8a91-87fa369b8d11 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.509698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf70a0-7079-4953-8a91-87fa369b8d11 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.511512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 371/738] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:28:59.511984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf70a0-7079-4953-8a91-87fa369b8d11 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:59.534158 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3380f-9887-463e-9251-a83a0088fae1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:59.567647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d51a1f-5d37-4715-9742-0f8320346915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:59.568311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d51a1f-5d37-4715-9742-0f8320346915 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.568592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d51a1f-5d37-4715-9742-0f8320346915 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.568878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d51a1f-5d37-4715-9742-0f8320346915 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.569598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d51a1f-5d37-4715-9742-0f8320346915 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:59.570979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d51a1f-5d37-4715-9742-0f8320346915 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:59.581361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:59.599488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 368/739] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:28:59 2022] POST /identity/v3/services => generated 251 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:28:59.599488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7543902-ca51-4252-9791-36645b7b69fe None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.599488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7543902-ca51-4252-9791-36645b7b69fe None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.599488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7543902-ca51-4252-9791-36645b7b69fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.619694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c7543902-ca51-4252-9791-36645b7b69fe None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:59.620993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c7543902-ca51-4252-9791-36645b7b69fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:59.631065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf70a0-7079-4953-8a91-87fa369b8d11 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:59.668131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393cbc6-8f3a-443b-bb54-a6e82e1d3f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:59.668959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393cbc6-8f3a-443b-bb54-a6e82e1d3f10 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.669246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393cbc6-8f3a-443b-bb54-a6e82e1d3f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.669528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393cbc6-8f3a-443b-bb54-a6e82e1d3f10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.679022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393cbc6-8f3a-443b-bb54-a6e82e1d3f10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:59.680646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393cbc6-8f3a-443b-bb54-a6e82e1d3f10 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5393cbc6-8f3a-443b-bb54-a6e82e1d3f10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:28:59.695955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:59.697602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:28:59.698993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:59.703184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 372/740] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:59.704222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393cbc6-8f3a-443b-bb54-a6e82e1d3f10 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:59.714490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 369/741] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:59.714490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d112ce5-0b89-4b75-909a-591bceaf945c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.714490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d112ce5-0b89-4b75-909a-591bceaf945c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.714490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d112ce5-0b89-4b75-909a-591bceaf945c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.738084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6d112ce5-0b89-4b75-909a-591bceaf945c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:28:59.739288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6d112ce5-0b89-4b75-909a-591bceaf945c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:28:59.747402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:59.819572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 370/742] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:28:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:28:59.820883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d112ce5-0b89-4b75-909a-591bceaf945c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:59.836166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393cbc6-8f3a-443b-bb54-a6e82e1d3f10 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:59.870331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05928a-ff94-4060-9b3b-788ba8da9470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:59.871178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05928a-ff94-4060-9b3b-788ba8da9470 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.871486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05928a-ff94-4060-9b3b-788ba8da9470 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.871778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05928a-ff94-4060-9b3b-788ba8da9470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.886063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:28:59.886636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05928a-ff94-4060-9b3b-788ba8da9470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:59.888163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05928a-ff94-4060-9b3b-788ba8da9470 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0a05928a-ff94-4060-9b3b-788ba8da9470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:28:59.908295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:28:59.909230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:28:59.910102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:28:59.910102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:28:59.910102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:28:59.910102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:28:59.910726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 373/743] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:28:59.911254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05928a-ff94-4060-9b3b-788ba8da9470 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:28:59.931369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d112ce5-0b89-4b75-909a-591bceaf945c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:28:59.970318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd126045-59b9-475c-9c08-b192adcc2151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:28:59.971239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd126045-59b9-475c-9c08-b192adcc2151 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:28:59.971727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd126045-59b9-475c-9c08-b192adcc2151 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:28:59.972008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd126045-59b9-475c-9c08-b192adcc2151 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:28:59.972837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd126045-59b9-475c-9c08-b192adcc2151 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:28:59.973231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd126045-59b9-475c-9c08-b192adcc2151 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:28:59.974594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd126045-59b9-475c-9c08-b192adcc2151 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:28:59.993075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 371/744] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 13 20:28:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:28:59.994029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd126045-59b9-475c-9c08-b192adcc2151 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:00.004819 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:00.026011 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05928a-ff94-4060-9b3b-788ba8da9470 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:00.067674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89c8c9-0bf2-46bd-9dca-673931103a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:00.068403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89c8c9-0bf2-46bd-9dca-673931103a2a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:00.069336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89c8c9-0bf2-46bd-9dca-673931103a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:00.069336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89c8c9-0bf2-46bd-9dca-673931103a2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:00.080405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89c8c9-0bf2-46bd-9dca-673931103a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0d89c8c9-0bf2-46bd-9dca-673931103a2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:00.088124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:00.089265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 374/745] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:28:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:00.095181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89c8c9-0bf2-46bd-9dca-673931103a2a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:00.108154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd126045-59b9-475c-9c08-b192adcc2151 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:00.127118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:00.144635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc842092-684c-4d8e-88f1-3061bcf62b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:00.145316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc842092-684c-4d8e-88f1-3061bcf62b8f None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:00.145608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc842092-684c-4d8e-88f1-3061bcf62b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:00.145896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc842092-684c-4d8e-88f1-3061bcf62b8f None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:00.146617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc842092-684c-4d8e-88f1-3061bcf62b8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:00.148049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc842092-684c-4d8e-88f1-3061bcf62b8f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:00.175522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 372/746] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:28:59 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:00.175522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc842092-684c-4d8e-88f1-3061bcf62b8f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:00.195816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89c8c9-0bf2-46bd-9dca-673931103a2a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:00.248381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b1907-8e74-4319-a2a7-1e7c4cf89875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:00.249207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b1907-8e74-4319-a2a7-1e7c4cf89875 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:00.249595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b1907-8e74-4319-a2a7-1e7c4cf89875 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:00.250175 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b1907-8e74-4319-a2a7-1e7c4cf89875 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:00.251131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b1907-8e74-4319-a2a7-1e7c4cf89875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:00.251568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b1907-8e74-4319-a2a7-1e7c4cf89875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:00.253257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b1907-8e74-4319-a2a7-1e7c4cf89875 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:00.256266 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:00.269168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 375/747] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:00.270431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b1907-8e74-4319-a2a7-1e7c4cf89875 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:00.301718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc842092-684c-4d8e-88f1-3061bcf62b8f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:00.338295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508a8ef-907f-4921-8886-3dc6deb2842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:00.338873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508a8ef-907f-4921-8886-3dc6deb2842b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:00.339286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508a8ef-907f-4921-8886-3dc6deb2842b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:00.339612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508a8ef-907f-4921-8886-3dc6deb2842b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:00.388983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b1907-8e74-4319-a2a7-1e7c4cf89875 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:00.394873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:00.427138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508a8ef-907f-4921-8886-3dc6deb2842b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:00.428958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508a8ef-907f-4921-8886-3dc6deb2842b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:00.538696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957e9c9-e1c8-4a5d-88c2-830a0399a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:00.539383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957e9c9-e1c8-4a5d-88c2-830a0399a017 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:00.539744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957e9c9-e1c8-4a5d-88c2-830a0399a017 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:00.540030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957e9c9-e1c8-4a5d-88c2-830a0399a017 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:00.540815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957e9c9-e1c8-4a5d-88c2-830a0399a017 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:00.541245 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957e9c9-e1c8-4a5d-88c2-830a0399a017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:00.542758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957e9c9-e1c8-4a5d-88c2-830a0399a017 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:00.554652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 376/749] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:29:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:00.554652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957e9c9-e1c8-4a5d-88c2-830a0399a017 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:00.555099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 373/749] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:00.556226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508a8ef-907f-4921-8886-3dc6deb2842b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:00.624001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:00.655990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:00.662471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957e9c9-e1c8-4a5d-88c2-830a0399a017 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:00.679805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508a8ef-907f-4921-8886-3dc6deb2842b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:00.710243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f274aa3-405c-4577-90b1-23a20fe429af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:00.712543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f274aa3-405c-4577-90b1-23a20fe429af None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:00.713305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f274aa3-405c-4577-90b1-23a20fe429af None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:00.713787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f274aa3-405c-4577-90b1-23a20fe429af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:00.717032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc72b6-5252-4e75-a7d5-d62db2a0151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:00.717933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc72b6-5252-4e75-a7d5-d62db2a0151b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:00.718216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc72b6-5252-4e75-a7d5-d62db2a0151b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:00.718490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc72b6-5252-4e75-a7d5-d62db2a0151b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:00.723723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f274aa3-405c-4577-90b1-23a20fe429af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:00.725921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f274aa3-405c-4577-90b1-23a20fe429af None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:00.729166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc72b6-5252-4e75-a7d5-d62db2a0151b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:00.730785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc72b6-5252-4e75-a7d5-d62db2a0151b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-6f274aa3-405c-4577-90b1-23a20fe429af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:00.743831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:00.744742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:00.745692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:00.745692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:00.745692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:00.745692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:00.745692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:00.745692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:00.745692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:00.745692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:00.748193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 377/750] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-cebc72b6-5252-4e75-a7d5-d62db2a0151b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:00.748487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:00.749314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:00.750198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:00.750198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:00.750198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:00.750198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:00.751552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f274aa3-405c-4577-90b1-23a20fe429af None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:00.754278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 374/751] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:00.755107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc72b6-5252-4e75-a7d5-d62db2a0151b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:00.879508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc72b6-5252-4e75-a7d5-d62db2a0151b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:00.880997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f274aa3-405c-4577-90b1-23a20fe429af None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:00.902823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:00.913290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:00.920592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961023b-d721-44ab-bbd5-8187b4e744a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:00.921267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961023b-d721-44ab-bbd5-8187b4e744a1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:00.921559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961023b-d721-44ab-bbd5-8187b4e744a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:00.921853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961023b-d721-44ab-bbd5-8187b4e744a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:00.922632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961023b-d721-44ab-bbd5-8187b4e744a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:00.922947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961023b-d721-44ab-bbd5-8187b4e744a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:00.924004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41000f-0798-4117-aa46-dc1b63503d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:00.924427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961023b-d721-44ab-bbd5-8187b4e744a1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:00.925405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41000f-0798-4117-aa46-dc1b63503d8d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:00.925898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41000f-0798-4117-aa46-dc1b63503d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:00.926363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41000f-0798-4117-aa46-dc1b63503d8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:00.937258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41000f-0798-4117-aa46-dc1b63503d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:00.939013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41000f-0798-4117-aa46-dc1b63503d8d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:00.942685 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 375/752] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:00.943607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961023b-d721-44ab-bbd5-8187b4e744a1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fc41000f-0798-4117-aa46-dc1b63503d8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:00.958275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:00.959170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:00.960085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:00.960085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:00.960085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:00.960085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:00.961977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 378/753] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:00.962956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41000f-0798-4117-aa46-dc1b63503d8d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:01.056674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961023b-d721-44ab-bbd5-8187b4e744a1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:01.069772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41000f-0798-4117-aa46-dc1b63503d8d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:01.097062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09312bb-9660-4941-b35c-1fb94af507be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:01.098083 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09312bb-9660-4941-b35c-1fb94af507be None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.098515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09312bb-9660-4941-b35c-1fb94af507be None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.098935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09312bb-9660-4941-b35c-1fb94af507be None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.099837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09312bb-9660-4941-b35c-1fb94af507be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:01.100384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09312bb-9660-4941-b35c-1fb94af507be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:01.102089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09312bb-9660-4941-b35c-1fb94af507be None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:01.106210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534749da-6474-4468-955c-53b2b1dcc71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:01.106982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534749da-6474-4468-955c-53b2b1dcc71b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.107377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534749da-6474-4468-955c-53b2b1dcc71b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.107786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534749da-6474-4468-955c-53b2b1dcc71b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.108712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534749da-6474-4468-955c-53b2b1dcc71b None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=95dcfd3d412a47298d7d5cf7b3b56f97) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:01.109131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534749da-6474-4468-955c-53b2b1dcc71b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:01.110642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534749da-6474-4468-955c-53b2b1dcc71b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:01.119573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 379/754] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:00 2022] GET /identity/v3/role_assignments?user.id=95dcfd3d412a47298d7d5cf7b3b56f97&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:01.119573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534749da-6474-4468-955c-53b2b1dcc71b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:01.120157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 376/755] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:01.129173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09312bb-9660-4941-b35c-1fb94af507be None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:01.155957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:01.208658 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:01.228710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534749da-6474-4468-955c-53b2b1dcc71b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:01.269935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09312bb-9660-4941-b35c-1fb94af507be None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:01.308679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93b5d5-b343-462a-bde6-6ea35244b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:01.308892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77511a-ba36-497c-a494-8b36f60449b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:01.309366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77511a-ba36-497c-a494-8b36f60449b9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.309655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77511a-ba36-497c-a494-8b36f60449b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.309934 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77511a-ba36-497c-a494-8b36f60449b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.310615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93b5d5-b343-462a-bde6-6ea35244b3cd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.310729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77511a-ba36-497c-a494-8b36f60449b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:01.311073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77511a-ba36-497c-a494-8b36f60449b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:01.311401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93b5d5-b343-462a-bde6-6ea35244b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.311900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93b5d5-b343-462a-bde6-6ea35244b3cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.313144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77511a-ba36-497c-a494-8b36f60449b9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:01.320263 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93b5d5-b343-462a-bde6-6ea35244b3cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:01.322089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93b5d5-b343-462a-bde6-6ea35244b3cd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2c93b5d5-b343-462a-bde6-6ea35244b3cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:01.338758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:01.339909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:01.340782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:01.340782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:01.340782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:01.340782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:01.340782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:01.340782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:01.340782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:01.340782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:01.343293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 377/756] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:01.347552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 380/757] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:01.349289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77511a-ba36-497c-a494-8b36f60449b9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:01.350367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93b5d5-b343-462a-bde6-6ea35244b3cd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:01.469478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:01.479179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77511a-ba36-497c-a494-8b36f60449b9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:01.517924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38341106-2187-48ef-9653-cb2755adf874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:01.518979 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38341106-2187-48ef-9653-cb2755adf874 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.519492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38341106-2187-48ef-9653-cb2755adf874 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.519985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38341106-2187-48ef-9653-cb2755adf874 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.521629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38341106-2187-48ef-9653-cb2755adf874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:01.522171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38341106-2187-48ef-9653-cb2755adf874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:01.524130 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38341106-2187-48ef-9653-cb2755adf874 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:01.539646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 381/758] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:01.543550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93b5d5-b343-462a-bde6-6ea35244b3cd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:01.552049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38341106-2187-48ef-9653-cb2755adf874 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:01.563727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:01.601863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca096f-9f18-4466-861c-898d8ab1ec3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:01.603105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca096f-9f18-4466-861c-898d8ab1ec3a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.603752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca096f-9f18-4466-861c-898d8ab1ec3a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.604238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca096f-9f18-4466-861c-898d8ab1ec3a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.613276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca096f-9f18-4466-861c-898d8ab1ec3a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:01.615181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca096f-9f18-4466-861c-898d8ab1ec3a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-15ca096f-9f18-4466-861c-898d8ab1ec3a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:01.630725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:01.631721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:01.632670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:01.635501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 378/759] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 13 20:29:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:01.644050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca096f-9f18-4466-861c-898d8ab1ec3a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:01.753224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38341106-2187-48ef-9653-cb2755adf874 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:01.783928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca096f-9f18-4466-861c-898d8ab1ec3a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:01.797225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba443c-8de8-4a53-b901-2ac653a0076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:01.798170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba443c-8de8-4a53-b901-2ac653a0076c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.798578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba443c-8de8-4a53-b901-2ac653a0076c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.799354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba443c-8de8-4a53-b901-2ac653a0076c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.800652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba443c-8de8-4a53-b901-2ac653a0076c None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=3f27793f58c74ffb9a7a8e4cdddb8169) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:01.801075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba443c-8de8-4a53-b901-2ac653a0076c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:01.803008 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba443c-8de8-4a53-b901-2ac653a0076c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:01.810624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 382/760] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:01 2022] GET /identity/v3/role_assignments?user.id=3f27793f58c74ffb9a7a8e4cdddb8169&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:01.813051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d53838-a067-4d43-af5a-deaa132bf334 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.813343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d53838-a067-4d43-af5a-deaa132bf334 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.813625 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d53838-a067-4d43-af5a-deaa132bf334 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.816069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 383/761] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:01.821037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73f47b-6fee-4498-acc8-df418c8517c9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.821327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73f47b-6fee-4498-acc8-df418c8517c9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.821604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73f47b-6fee-4498-acc8-df418c8517c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.841592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02621b4b-ffe0-4c69-95f3-a73c63eca590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:01.842298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02621b4b-ffe0-4c69-95f3-a73c63eca590 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:01.842588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02621b4b-ffe0-4c69-95f3-a73c63eca590 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:01.842870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02621b4b-ffe0-4c69-95f3-a73c63eca590 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:01.843763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02621b4b-ffe0-4c69-95f3-a73c63eca590 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:01.844078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02621b4b-ffe0-4c69-95f3-a73c63eca590 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:01.845596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02621b4b-ffe0-4c69-95f3-a73c63eca590 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:01.860991 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6a73f47b-6fee-4498-acc8-df418c8517c9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:01.862231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6a73f47b-6fee-4498-acc8-df418c8517c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:01.884011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 379/762] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 13 20:29:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:01.887893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02621b4b-ffe0-4c69-95f3-a73c63eca590 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:01.899345 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:01.934520 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:01.999516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 384/763] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:02.032621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73f47b-6fee-4498-acc8-df418c8517c9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:02.055022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02621b4b-ffe0-4c69-95f3-a73c63eca590 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:02.104190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc5247-9547-4ac2-89f9-a1c7ea03f02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:02.105038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc5247-9547-4ac2-89f9-a1c7ea03f02f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.105320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc5247-9547-4ac2-89f9-a1c7ea03f02f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.105595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc5247-9547-4ac2-89f9-a1c7ea03f02f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.132582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc5247-9547-4ac2-89f9-a1c7ea03f02f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:02.134060 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc5247-9547-4ac2-89f9-a1c7ea03f02f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:02.179347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73f47b-6fee-4498-acc8-df418c8517c9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9fcc5247-9547-4ac2-89f9-a1c7ea03f02f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:02.180776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:02.181635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:02.182998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:02.182998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:02.182998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:02.182998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:02.182998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 380/764] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:02.189049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc5247-9547-4ac2-89f9-a1c7ea03f02f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:02.220759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042e830-8ead-4a66-bc4a-bbbc18e758c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:02.221589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042e830-8ead-4a66-bc4a-bbbc18e758c4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.221868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042e830-8ead-4a66-bc4a-bbbc18e758c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.222141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042e830-8ead-4a66-bc4a-bbbc18e758c4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.222797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042e830-8ead-4a66-bc4a-bbbc18e758c4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:02.224214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042e830-8ead-4a66-bc4a-bbbc18e758c4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7042e830-8ead-4a66-bc4a-bbbc18e758c4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:02.259766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:02.262206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:02.264233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 385/765] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Jul 13 20:29:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:02.271764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042e830-8ead-4a66-bc4a-bbbc18e758c4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:02.299313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:02.309348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:02.323862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc5247-9547-4ac2-89f9-a1c7ea03f02f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:02.367887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572bfee-2eab-416f-819a-f6ad848b386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:02.368562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572bfee-2eab-416f-819a-f6ad848b386b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.368828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572bfee-2eab-416f-819a-f6ad848b386b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.369116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572bfee-2eab-416f-819a-f6ad848b386b None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.369845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572bfee-2eab-416f-819a-f6ad848b386b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:02.370181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572bfee-2eab-416f-819a-f6ad848b386b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:02.371596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572bfee-2eab-416f-819a-f6ad848b386b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:02.395136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 381/766] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:02.396951 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5a701-c086-4e1a-aa46-bbf54701886d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.397252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5a701-c086-4e1a-aa46-bbf54701886d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.397856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5a701-c086-4e1a-aa46-bbf54701886d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.399653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 382/767] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:02.401617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5a701-c086-4e1a-aa46-bbf54701886d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:02.448514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042e830-8ead-4a66-bc4a-bbbc18e758c4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:02.484324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de50cb-6be7-4bbd-9fae-4aa90ac5bb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:02.484989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de50cb-6be7-4bbd-9fae-4aa90ac5bb4c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.485384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de50cb-6be7-4bbd-9fae-4aa90ac5bb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.485659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de50cb-6be7-4bbd-9fae-4aa90ac5bb4c None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.486470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de50cb-6be7-4bbd-9fae-4aa90ac5bb4c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:02.486772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de50cb-6be7-4bbd-9fae-4aa90ac5bb4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:02.488291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de50cb-6be7-4bbd-9fae-4aa90ac5bb4c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:02.494692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 386/768] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 13 20:29:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:02.496518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b972b25-e614-4e2a-8dc6-2a1c309335b2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.496821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b972b25-e614-4e2a-8dc6-2a1c309335b2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.497105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b972b25-e614-4e2a-8dc6-2a1c309335b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.514087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8b972b25-e614-4e2a-8dc6-2a1c309335b2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:02.515272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8b972b25-e614-4e2a-8dc6-2a1c309335b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:02.523798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5a701-c086-4e1a-aa46-bbf54701886d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:02.560319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044a416-4208-4cd1-9335-e0950b7c102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:02.561507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044a416-4208-4cd1-9335-e0950b7c102f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.561995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044a416-4208-4cd1-9335-e0950b7c102f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.562486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044a416-4208-4cd1-9335-e0950b7c102f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.569949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044a416-4208-4cd1-9335-e0950b7c102f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:02.571832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044a416-4208-4cd1-9335-e0950b7c102f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:02.586712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 387/769] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-6044a416-4208-4cd1-9335-e0950b7c102f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:02.587189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:02.588320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:02.589264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:02.589264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:02.589264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:02.589264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:02.589264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:02.589264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:02.589264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:02.589264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:02.589689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b972b25-e614-4e2a-8dc6-2a1c309335b2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:02.595939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:02.596236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 383/770] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:02.596361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:02.597852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044a416-4208-4cd1-9335-e0950b7c102f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:02.693385 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b972b25-e614-4e2a-8dc6-2a1c309335b2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:02.704583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044a416-4208-4cd1-9335-e0950b7c102f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:02.729086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577da8f-e094-4b69-9bd0-d728a907b545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:02.729709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577da8f-e094-4b69-9bd0-d728a907b545 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.729998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577da8f-e094-4b69-9bd0-d728a907b545 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.730276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577da8f-e094-4b69-9bd0-d728a907b545 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.730983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577da8f-e094-4b69-9bd0-d728a907b545 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:02.731292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577da8f-e094-4b69-9bd0-d728a907b545 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:02.732960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577da8f-e094-4b69-9bd0-d728a907b545 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:02.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 388/771] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 13 20:29:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:02.739217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577da8f-e094-4b69-9bd0-d728a907b545 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:02.741192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836b613-5395-4b98-984b-bfac2612b06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:02.742085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836b613-5395-4b98-984b-bfac2612b06e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.742968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836b613-5395-4b98-984b-bfac2612b06e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.743451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836b613-5395-4b98-984b-bfac2612b06e None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.744472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836b613-5395-4b98-984b-bfac2612b06e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:02.744981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836b613-5395-4b98-984b-bfac2612b06e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:02.746685 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836b613-5395-4b98-984b-bfac2612b06e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:02.762135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 384/772] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:02.766835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836b613-5395-4b98-984b-bfac2612b06e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:02.825843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:02.835767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:02.841810 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577da8f-e094-4b69-9bd0-d728a907b545 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:02.867934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836b613-5395-4b98-984b-bfac2612b06e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:02.877347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe430a-6068-49fc-b953-f000b65f613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:02.878159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe430a-6068-49fc-b953-f000b65f613c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.878443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe430a-6068-49fc-b953-f000b65f613c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.878722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe430a-6068-49fc-b953-f000b65f613c None admin] PATH_INFO: `/v3/services/network` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.879492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe430a-6068-49fc-b953-f000b65f613c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:02.881122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe430a-6068-49fc-b953-f000b65f613c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-18fe430a-6068-49fc-b953-f000b65f613c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:02.893568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:02.894582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:29:02.895410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:02.899721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 389/773] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:02 2022] GET /identity/v3/services/network => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:02.900674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe430a-6068-49fc-b953-f000b65f613c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:02.909613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b0ee6-92bf-4a8d-81fa-943288edee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:02.910927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b0ee6-92bf-4a8d-81fa-943288edee44 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:02.911422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b0ee6-92bf-4a8d-81fa-943288edee44 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:02.911949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b0ee6-92bf-4a8d-81fa-943288edee44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:02.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b0ee6-92bf-4a8d-81fa-943288edee44 None admin] RBAC: Adding query filter params (interface=public, service_id=ad66b9f5f5454105b962247c291684de, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:02.913491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b0ee6-92bf-4a8d-81fa-943288edee44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:02.915100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b0ee6-92bf-4a8d-81fa-943288edee44 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:02.923376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 385/774] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 13 20:29:02 2022] GET /identity/v3/endpoints?service_id=ad66b9f5f5454105b962247c291684de&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:02.924588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b0ee6-92bf-4a8d-81fa-943288edee44 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:03.007992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe430a-6068-49fc-b953-f000b65f613c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:03.028320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b0ee6-92bf-4a8d-81fa-943288edee44 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:03.044225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c4fcd-b2c2-4654-827c-277807d0c5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:03.045126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c4fcd-b2c2-4654-827c-277807d0c5cb None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:03.045412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c4fcd-b2c2-4654-827c-277807d0c5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:03.045690 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c4fcd-b2c2-4654-827c-277807d0c5cb None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/e80bb5b10ce945cabbfb88ca5df92e3c/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:03.048418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c4fcd-b2c2-4654-827c-277807d0c5cb None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=e80bb5b10ce945cabbfb88ca5df92e3c, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:03.049881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c4fcd-b2c2-4654-827c-277807d0c5cb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:03.054883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:03.059795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:03.067706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46af211-68b5-4ba6-ad05-675f668fd456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:03.068518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46af211-68b5-4ba6-ad05-675f668fd456 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:03.069200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46af211-68b5-4ba6-ad05-675f668fd456 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:03.069603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46af211-68b5-4ba6-ad05-675f668fd456 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:03.070597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46af211-68b5-4ba6-ad05-675f668fd456 None admin] RBAC: Adding query filter params (name=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:03.071089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46af211-68b5-4ba6-ad05-675f668fd456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:03.072628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46af211-68b5-4ba6-ad05-675f668fd456 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:03.075009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 390/775] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:29:02 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/e80bb5b10ce945cabbfb88ca5df92e3c/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:29:03.076103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c4fcd-b2c2-4654-827c-277807d0c5cb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:03.083785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 386/776] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:03.094204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46af211-68b5-4ba6-ad05-675f668fd456 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:03.259256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46af211-68b5-4ba6-ad05-675f668fd456 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:03.290388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c4fcd-b2c2-4654-827c-277807d0c5cb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:03.304594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09428a35-b561-4e31-9654-9e8f4a4cbfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:03.305511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09428a35-b561-4e31-9654-9e8f4a4cbfe8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:03.307287 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09428a35-b561-4e31-9654-9e8f4a4cbfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:03.307831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09428a35-b561-4e31-9654-9e8f4a4cbfe8 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:03.308694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09428a35-b561-4e31-9654-9e8f4a4cbfe8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:03.309151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09428a35-b561-4e31-9654-9e8f4a4cbfe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:03.311490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09428a35-b561-4e31-9654-9e8f4a4cbfe8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:03.327174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 387/777] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:03.331036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb6ab0-f1be-41ff-b0c2-a5be647109a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:03.332110 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb6ab0-f1be-41ff-b0c2-a5be647109a9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:03.332396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb6ab0-f1be-41ff-b0c2-a5be647109a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:03.332723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb6ab0-f1be-41ff-b0c2-a5be647109a9 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:03.333413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb6ab0-f1be-41ff-b0c2-a5be647109a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:03.334155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09428a35-b561-4e31-9654-9e8f4a4cbfe8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:03.334936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb6ab0-f1be-41ff-b0c2-a5be647109a9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:03.341125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 391/778] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 13 20:29:03 2022] GET /identity/v3/services => generated 772 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:03.401900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:03.471931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09428a35-b561-4e31-9654-9e8f4a4cbfe8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:03.522766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ff61-305f-472c-84a1-44a87ac518bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:03.523451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ff61-305f-472c-84a1-44a87ac518bf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:03.523755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ff61-305f-472c-84a1-44a87ac518bf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:03.524031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ff61-305f-472c-84a1-44a87ac518bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:03.524836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08ff61-305f-472c-84a1-44a87ac518bf None admin] RBAC: Adding query filter params (interface=public, service_id=951db8e260c54b78ba385cac99ed0cef, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:03.525151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08ff61-305f-472c-84a1-44a87ac518bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:03.526687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08ff61-305f-472c-84a1-44a87ac518bf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:03.530423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ecdb4-7c9b-40c7-903a-ce5c6d2005cf None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:03.530718 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ecdb4-7c9b-40c7-903a-ce5c6d2005cf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:03.531002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ecdb4-7c9b-40c7-903a-ce5c6d2005cf None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:03.532973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 388/780] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 13 20:29:03 2022] GET /identity/v3/endpoints?service_id=951db8e260c54b78ba385cac99ed0cef&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:03.536569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 392/780] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:03.538206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84afe68-1f6b-4c23-aa38-2a8b33c38d1f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:03.538498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84afe68-1f6b-4c23-aa38-2a8b33c38d1f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:03.538619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ecdb4-7c9b-40c7-903a-ce5c6d2005cf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:03.538816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84afe68-1f6b-4c23-aa38-2a8b33c38d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:03.557214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c84afe68-1f6b-4c23-aa38-2a8b33c38d1f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:03.558575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c84afe68-1f6b-4c23-aa38-2a8b33c38d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:03.609225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:03.643951 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 389/781] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:03.661340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ecdb4-7c9b-40c7-903a-ce5c6d2005cf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:03.664229 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84afe68-1f6b-4c23-aa38-2a8b33c38d1f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:03.703859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f759e-25b4-4f56-bb62-6d9783883bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:03.704597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f759e-25b4-4f56-bb62-6d9783883bfb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:03.704882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f759e-25b4-4f56-bb62-6d9783883bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:03.705206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f759e-25b4-4f56-bb62-6d9783883bfb None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:03.705903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f759e-25b4-4f56-bb62-6d9783883bfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:03.707775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f759e-25b4-4f56-bb62-6d9783883bfb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:03.716952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 393/782] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 13 20:29:03 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:03.724995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:03.821540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84afe68-1f6b-4c23-aa38-2a8b33c38d1f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:03.873092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ba9e9-d80a-40c6-aaff-b93d30adaa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:03.874260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ba9e9-d80a-40c6-aaff-b93d30adaa41 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:03.874733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ba9e9-d80a-40c6-aaff-b93d30adaa41 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:03.875211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ba9e9-d80a-40c6-aaff-b93d30adaa41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:03.896551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ba9e9-d80a-40c6-aaff-b93d30adaa41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1f4ba9e9-d80a-40c6-aaff-b93d30adaa41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:03.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:03.905789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:29:03.906619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:03.913825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 390/783] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:29:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:03.916570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f759e-25b4-4f56-bb62-6d9783883bfb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:04.020369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f759e-25b4-4f56-bb62-6d9783883bfb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:04.044887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:04.058741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53d740-4ccc-4058-b24b-eb63bb580060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:04.059432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d740-4ccc-4058-b24b-eb63bb580060 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:04.059750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d740-4ccc-4058-b24b-eb63bb580060 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:04.060029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d740-4ccc-4058-b24b-eb63bb580060 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:04.060791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53d740-4ccc-4058-b24b-eb63bb580060 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:04.061101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53d740-4ccc-4058-b24b-eb63bb580060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:04.062493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53d740-4ccc-4058-b24b-eb63bb580060 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:04.071569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 394/784] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:29:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:04.077577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ba9e9-d80a-40c6-aaff-b93d30adaa41 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:04.182216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33774d-8f11-47c3-9bb5-e4bb47d30aa9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:04.182918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33774d-8f11-47c3-9bb5-e4bb47d30aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:04.183403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33774d-8f11-47c3-9bb5-e4bb47d30aa9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:04.186594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 395/785] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:04.191503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b208c21-053f-469a-a7f3-116b4a5f236c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:04.192094 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b208c21-053f-469a-a7f3-116b4a5f236c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:04.192573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b208c21-053f-469a-a7f3-116b4a5f236c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:04.230336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3b208c21-053f-469a-a7f3-116b4a5f236c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:04.231556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3b208c21-053f-469a-a7f3-116b4a5f236c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:04.356647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 396/786] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:04.382360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ba9e9-d80a-40c6-aaff-b93d30adaa41 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:04.390473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:04.404896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b208c21-053f-469a-a7f3-116b4a5f236c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:04.437473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8ed4a-72df-4359-bfd1-3d1cfc60be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:04.438205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8ed4a-72df-4359-bfd1-3d1cfc60be15 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:04.438493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8ed4a-72df-4359-bfd1-3d1cfc60be15 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:04.438775 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8ed4a-72df-4359-bfd1-3d1cfc60be15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:04.453899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8ed4a-72df-4359-bfd1-3d1cfc60be15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:04.455377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8ed4a-72df-4359-bfd1-3d1cfc60be15 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c9c8ed4a-72df-4359-bfd1-3d1cfc60be15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:04.496239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:04.497166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:04.498047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:04.498047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:04.498047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:04.498047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:04.498047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 391/787] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:04.518367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8ed4a-72df-4359-bfd1-3d1cfc60be15 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:04.585971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b208c21-053f-469a-a7f3-116b4a5f236c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:04.603836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:04.630921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ee221-d85f-445c-88d0-47e9a5e57d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:04.631777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ee221-d85f-445c-88d0-47e9a5e57d60 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:04.632129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ee221-d85f-445c-88d0-47e9a5e57d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:04.632471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ee221-d85f-445c-88d0-47e9a5e57d60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:04.653270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ee221-d85f-445c-88d0-47e9a5e57d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-834ee221-d85f-445c-88d0-47e9a5e57d60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:04.661087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:04.662062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:04.662883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:04.678967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 397/788] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:04.695551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ee221-d85f-445c-88d0-47e9a5e57d60 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:04.727959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8ed4a-72df-4359-bfd1-3d1cfc60be15 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:04.803908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0133a1-7778-467a-b990-cb3301c3599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:04.804583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0133a1-7778-467a-b990-cb3301c3599c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:04.806248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0133a1-7778-467a-b990-cb3301c3599c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:04.806542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0133a1-7778-467a-b990-cb3301c3599c None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:04.807282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0133a1-7778-467a-b990-cb3301c3599c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:04.807660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0133a1-7778-467a-b990-cb3301c3599c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:04.809063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0133a1-7778-467a-b990-cb3301c3599c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:04.843655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 392/789] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:04.844943 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0133a1-7778-467a-b990-cb3301c3599c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:04.873888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ee221-d85f-445c-88d0-47e9a5e57d60 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:04.890888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:04.914771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ece763-1a56-443e-9492-4fa17cb014f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:04.915525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece763-1a56-443e-9492-4fa17cb014f3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:04.915948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece763-1a56-443e-9492-4fa17cb014f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:04.916235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece763-1a56-443e-9492-4fa17cb014f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:04.917025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ece763-1a56-443e-9492-4fa17cb014f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:04.917551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ece763-1a56-443e-9492-4fa17cb014f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:04.918703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ece763-1a56-443e-9492-4fa17cb014f3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:04.931609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 398/790] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:04.934529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ece763-1a56-443e-9492-4fa17cb014f3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:04.974617 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:05.043880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ece763-1a56-443e-9492-4fa17cb014f3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.080895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3089e-c65e-4352-a102-b1657dd99f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:05.081758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3089e-c65e-4352-a102-b1657dd99f42 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.082048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3089e-c65e-4352-a102-b1657dd99f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.082326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3089e-c65e-4352-a102-b1657dd99f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.093864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3089e-c65e-4352-a102-b1657dd99f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:05.095424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3089e-c65e-4352-a102-b1657dd99f42 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2ad3089e-c65e-4352-a102-b1657dd99f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:05.116871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:05.117851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:05.118732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:05.118732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:05.118732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:05.118732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:05.123475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 399/791] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:05.124676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3089e-c65e-4352-a102-b1657dd99f42 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:05.225495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0133a1-7778-467a-b990-cb3301c3599c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.230835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3089e-c65e-4352-a102-b1657dd99f42 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.247808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:05.272947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c711360-b3fb-4297-a28e-26d68c70a931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:05.273685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c711360-b3fb-4297-a28e-26d68c70a931 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.273979 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c711360-b3fb-4297-a28e-26d68c70a931 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.274254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c711360-b3fb-4297-a28e-26d68c70a931 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.274989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c711360-b3fb-4297-a28e-26d68c70a931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:05.275316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c711360-b3fb-4297-a28e-26d68c70a931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:05.281106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c711360-b3fb-4297-a28e-26d68c70a931 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:05.311573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 400/792] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:05.313243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7092cbe-d1b0-4f83-8e89-0e93609160e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:05.314139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7092cbe-d1b0-4f83-8e89-0e93609160e4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.314428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7092cbe-d1b0-4f83-8e89-0e93609160e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.314707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7092cbe-d1b0-4f83-8e89-0e93609160e4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.322610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7092cbe-d1b0-4f83-8e89-0e93609160e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:05.324543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7092cbe-d1b0-4f83-8e89-0e93609160e4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:05.327594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c711360-b3fb-4297-a28e-26d68c70a931 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e7092cbe-d1b0-4f83-8e89-0e93609160e4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:05.359851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:05.360858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:05.361802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 393/793] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:29:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 517 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:05.368407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7092cbe-d1b0-4f83-8e89-0e93609160e4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:05.432813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c711360-b3fb-4297-a28e-26d68c70a931 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.444776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:05.469235 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7092cbe-d1b0-4f83-8e89-0e93609160e4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.471141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9689c7c-1789-4dce-85ed-80819e0093b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:05.472122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9689c7c-1789-4dce-85ed-80819e0093b2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.472414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9689c7c-1789-4dce-85ed-80819e0093b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.472697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9689c7c-1789-4dce-85ed-80819e0093b2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.480765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9689c7c-1789-4dce-85ed-80819e0093b2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:05.482155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9689c7c-1789-4dce-85ed-80819e0093b2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a9689c7c-1789-4dce-85ed-80819e0093b2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:05.498158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:05.499124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:29:05.500087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:05.503833 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 401/794] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:29:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:05.505405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5abf52-ae83-449a-be87-4a8c9855e3b0 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.505692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5abf52-ae83-449a-be87-4a8c9855e3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.505971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5abf52-ae83-449a-be87-4a8c9855e3b0 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.507265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 402/795] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:05.507972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5abf52-ae83-449a-be87-4a8c9855e3b0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:05.511261 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969de2b5-b7f8-4e75-a08f-deae53bb2f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:05.511978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969de2b5-b7f8-4e75-a08f-deae53bb2f7f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.512258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969de2b5-b7f8-4e75-a08f-deae53bb2f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.512534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969de2b5-b7f8-4e75-a08f-deae53bb2f7f None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.513372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969de2b5-b7f8-4e75-a08f-deae53bb2f7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:05.513679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969de2b5-b7f8-4e75-a08f-deae53bb2f7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:05.515114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969de2b5-b7f8-4e75-a08f-deae53bb2f7f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:05.526047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:05.536488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 394/796] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 13 20:29:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:05.537788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb3a31-c10c-40e1-9361-0eeece001ace None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.538170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb3a31-c10c-40e1-9361-0eeece001ace None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.538444 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb3a31-c10c-40e1-9361-0eeece001ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.557094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a9bb3a31-c10c-40e1-9361-0eeece001ace None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:05.558335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a9bb3a31-c10c-40e1-9361-0eeece001ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:05.624305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5abf52-ae83-449a-be87-4a8c9855e3b0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.628950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 395/797] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:05.629812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb3a31-c10c-40e1-9361-0eeece001ace None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:05.685198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed058e-2451-4f11-bc75-b5fd17fab274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:05.685888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed058e-2451-4f11-bc75-b5fd17fab274 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.686269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed058e-2451-4f11-bc75-b5fd17fab274 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.686555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed058e-2451-4f11-bc75-b5fd17fab274 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.687596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed058e-2451-4f11-bc75-b5fd17fab274 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:05.687860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed058e-2451-4f11-bc75-b5fd17fab274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:05.689419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed058e-2451-4f11-bc75-b5fd17fab274 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:05.696270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:05.713732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 403/798] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 13 20:29:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:05.714683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed058e-2451-4f11-bc75-b5fd17fab274 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:05.732720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb3a31-c10c-40e1-9361-0eeece001ace None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.768008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e546bea-6b2f-4ae9-ac65-9c2ff388a77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:05.768913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e546bea-6b2f-4ae9-ac65-9c2ff388a77d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.769208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e546bea-6b2f-4ae9-ac65-9c2ff388a77d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.769485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e546bea-6b2f-4ae9-ac65-9c2ff388a77d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.782597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:05.784523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e546bea-6b2f-4ae9-ac65-9c2ff388a77d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:05.786039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e546bea-6b2f-4ae9-ac65-9c2ff388a77d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8e546bea-6b2f-4ae9-ac65-9c2ff388a77d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:05.803615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:05.805674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:05.807877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:05.807877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:05.807877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:05.807877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:05.815565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 396/799] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:05.815565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e546bea-6b2f-4ae9-ac65-9c2ff388a77d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:05.818152 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed058e-2451-4f11-bc75-b5fd17fab274 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.855053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d6208-b4b4-4d56-9e74-0dc1919619e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:05.855925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d6208-b4b4-4d56-9e74-0dc1919619e1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.856214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d6208-b4b4-4d56-9e74-0dc1919619e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.856492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d6208-b4b4-4d56-9e74-0dc1919619e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.868659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d6208-b4b4-4d56-9e74-0dc1919619e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3f5d6208-b4b4-4d56-9e74-0dc1919619e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:05.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:05.877466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:05.878270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:05.882522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 404/800] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:29:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:05.883268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d6208-b4b4-4d56-9e74-0dc1919619e1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:05.920173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e546bea-6b2f-4ae9-ac65-9c2ff388a77d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.939144 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:05.958211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c67e8-2da9-42ec-903f-f40b5a7a88ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:05.959424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c67e8-2da9-42ec-903f-f40b5a7a88ff None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.959424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c67e8-2da9-42ec-903f-f40b5a7a88ff None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.959653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c67e8-2da9-42ec-903f-f40b5a7a88ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.971442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c67e8-2da9-42ec-903f-f40b5a7a88ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:05.973282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c67e8-2da9-42ec-903f-f40b5a7a88ff None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:05.990830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d6208-b4b4-4d56-9e74-0dc1919619e1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4e2c67e8-2da9-42ec-903f-f40b5a7a88ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:05.991308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:05.992284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:05.993154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:05.993154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:05.993154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:05.993154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:05.993154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 397/801] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:05.993809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb334adc-a278-4e3c-ad3d-65a0dcaaaa6f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:05.993905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb334adc-a278-4e3c-ad3d-65a0dcaaaa6f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:05.994150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb334adc-a278-4e3c-ad3d-65a0dcaaaa6f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:05.999537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 398/802] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:05.999537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb334adc-a278-4e3c-ad3d-65a0dcaaaa6f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.025533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:06.035333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7418a-5e27-4d89-9798-4de1b40defb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.036048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7418a-5e27-4d89-9798-4de1b40defb3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.036332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7418a-5e27-4d89-9798-4de1b40defb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.036609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7418a-5e27-4d89-9798-4de1b40defb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.037334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7418a-5e27-4d89-9798-4de1b40defb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:06.037643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7418a-5e27-4d89-9798-4de1b40defb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.039013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7418a-5e27-4d89-9798-4de1b40defb3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.051496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 405/803] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:06.056778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7418a-5e27-4d89-9798-4de1b40defb3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.103875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb334adc-a278-4e3c-ad3d-65a0dcaaaa6f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.136819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd1534-85df-49f1-a0bf-6eeaaeefbc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.137480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd1534-85df-49f1-a0bf-6eeaaeefbc61 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.137765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd1534-85df-49f1-a0bf-6eeaaeefbc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.138039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd1534-85df-49f1-a0bf-6eeaaeefbc61 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.138768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd1534-85df-49f1-a0bf-6eeaaeefbc61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:06.139069 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd1534-85df-49f1-a0bf-6eeaaeefbc61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.140473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd1534-85df-49f1-a0bf-6eeaaeefbc61 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.148423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 399/804] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:29:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:06.149576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f529c95-7409-46ed-96f9-630b3a373ed7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.149861 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f529c95-7409-46ed-96f9-630b3a373ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.150126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f529c95-7409-46ed-96f9-630b3a373ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.159782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7418a-5e27-4d89-9798-4de1b40defb3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.166044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3f529c95-7409-46ed-96f9-630b3a373ed7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:06.167312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3f529c95-7409-46ed-96f9-630b3a373ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:06.192439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:06.193078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c7f14-e3a6-46ce-9e4b-fa6577563351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.193739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c7f14-e3a6-46ce-9e4b-fa6577563351 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.194058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c7f14-e3a6-46ce-9e4b-fa6577563351 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.194311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c7f14-e3a6-46ce-9e4b-fa6577563351 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.195004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c7f14-e3a6-46ce-9e4b-fa6577563351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:06.195310 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c7f14-e3a6-46ce-9e4b-fa6577563351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.196708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c7f14-e3a6-46ce-9e4b-fa6577563351 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.208311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 406/805] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:06.209104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c7f14-e3a6-46ce-9e4b-fa6577563351 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.235485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 400/806] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:06.236891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcf4e6-4d0b-4899-8793-056390a64960 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.237245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcf4e6-4d0b-4899-8793-056390a64960 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.237608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcf4e6-4d0b-4899-8793-056390a64960 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.238928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 401/807] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:06.239648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcf4e6-4d0b-4899-8793-056390a64960 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.269592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:06.326756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c7f14-e3a6-46ce-9e4b-fa6577563351 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.345318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcf4e6-4d0b-4899-8793-056390a64960 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.359246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ac0da-4156-461a-b1f4-ee8121de2421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.360086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ac0da-4156-461a-b1f4-ee8121de2421 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.360372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ac0da-4156-461a-b1f4-ee8121de2421 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.360652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ac0da-4156-461a-b1f4-ee8121de2421 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.367242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ac0da-4156-461a-b1f4-ee8121de2421 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.368724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ac0da-4156-461a-b1f4-ee8121de2421 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.378051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47489466-a6d7-422e-8fab-7bb1231d89c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.378822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47489466-a6d7-422e-8fab-7bb1231d89c9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.379142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47489466-a6d7-422e-8fab-7bb1231d89c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.379417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47489466-a6d7-422e-8fab-7bb1231d89c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-6d0ac0da-4156-461a-b1f4-ee8121de2421 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:06.382847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:06.383731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:06.384629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:06.384629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:06.384629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:06.384629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:06.384629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:06.384629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:06.384629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:06.384629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:06.384629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 407/808] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:06.385100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ac0da-4156-461a-b1f4-ee8121de2421 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.392709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47489466-a6d7-422e-8fab-7bb1231d89c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.394875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47489466-a6d7-422e-8fab-7bb1231d89c9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-47489466-a6d7-422e-8fab-7bb1231d89c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:06.412297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:06.413362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:06.414434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:06.414434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:06.414434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:06.414434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:06.414434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 402/809] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:06.414860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d496a-cf34-4294-b7d2-0c92d33a47ba None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.414978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d496a-cf34-4294-b7d2-0c92d33a47ba None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.415256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d496a-cf34-4294-b7d2-0c92d33a47ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.431016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-205d496a-cf34-4294-b7d2-0c92d33a47ba None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:06.432249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-205d496a-cf34-4294-b7d2-0c92d33a47ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:06.441500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:06.488606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ac0da-4156-461a-b1f4-ee8121de2421 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.496710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:06.497455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 403/810] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:06.498165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d496a-cf34-4294-b7d2-0c92d33a47ba None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.537782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4695949-d6e8-457f-8b4c-5ca3dbc021be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.538504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4695949-d6e8-457f-8b4c-5ca3dbc021be None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.538842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4695949-d6e8-457f-8b4c-5ca3dbc021be None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.539125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4695949-d6e8-457f-8b4c-5ca3dbc021be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.545799 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4695949-d6e8-457f-8b4c-5ca3dbc021be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.547254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4695949-d6e8-457f-8b4c-5ca3dbc021be None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f4695949-d6e8-457f-8b4c-5ca3dbc021be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:06.561318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:06.562352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 408/811] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:06.563519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4695949-d6e8-457f-8b4c-5ca3dbc021be None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.597632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d496a-cf34-4294-b7d2-0c92d33a47ba None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.633736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff857b6-1450-4c8e-8d9a-e0b7d3b8643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.634575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff857b6-1450-4c8e-8d9a-e0b7d3b8643f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.634859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff857b6-1450-4c8e-8d9a-e0b7d3b8643f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.635116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff857b6-1450-4c8e-8d9a-e0b7d3b8643f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.635926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff857b6-1450-4c8e-8d9a-e0b7d3b8643f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.637295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff857b6-1450-4c8e-8d9a-e0b7d3b8643f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-6ff857b6-1450-4c8e-8d9a-e0b7d3b8643f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:06.649414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:06.650420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:06.651337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 404/812] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:06.652781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff857b6-1450-4c8e-8d9a-e0b7d3b8643f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.662216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:06.673896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4695949-d6e8-457f-8b4c-5ca3dbc021be None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.710989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74abf8-4087-49db-a0ff-2e05f7cfa8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.711683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74abf8-4087-49db-a0ff-2e05f7cfa8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.711968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74abf8-4087-49db-a0ff-2e05f7cfa8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.712229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74abf8-4087-49db-a0ff-2e05f7cfa8c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.713010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74abf8-4087-49db-a0ff-2e05f7cfa8c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:06.713339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74abf8-4087-49db-a0ff-2e05f7cfa8c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.714885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74abf8-4087-49db-a0ff-2e05f7cfa8c7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.726767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 409/813] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:06.727645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74abf8-4087-49db-a0ff-2e05f7cfa8c7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.753881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff857b6-1450-4c8e-8d9a-e0b7d3b8643f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.754835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:06.787130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c990da5-f94f-4650-af6a-02629fcd942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.787893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c990da5-f94f-4650-af6a-02629fcd942e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.788120 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c990da5-f94f-4650-af6a-02629fcd942e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.788403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c990da5-f94f-4650-af6a-02629fcd942e None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.789127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c990da5-f94f-4650-af6a-02629fcd942e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:06.789439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c990da5-f94f-4650-af6a-02629fcd942e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.790831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c990da5-f94f-4650-af6a-02629fcd942e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.803534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 405/814] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:06.804337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c990da5-f94f-4650-af6a-02629fcd942e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.832954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74abf8-4087-49db-a0ff-2e05f7cfa8c7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.866785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aed60b-1a9b-4509-a75c-d17b196caeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.867621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aed60b-1a9b-4509-a75c-d17b196caeee None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.867910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aed60b-1a9b-4509-a75c-d17b196caeee None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.868183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aed60b-1a9b-4509-a75c-d17b196caeee None admin] PATH_INFO: `/v3/services/network` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.868853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aed60b-1a9b-4509-a75c-d17b196caeee None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.868981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:06.870309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aed60b-1a9b-4509-a75c-d17b196caeee None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-09aed60b-1a9b-4509-a75c-d17b196caeee None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:06.882005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:06.883134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:06.884096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 410/815] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/services/network => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:06.884942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aed60b-1a9b-4509-a75c-d17b196caeee None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.904557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c990da5-f94f-4650-af6a-02629fcd942e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:06.937428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2d67a-53df-4c4b-893d-735bc95d6823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:06.938082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2d67a-53df-4c4b-893d-735bc95d6823 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:06.938362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2d67a-53df-4c4b-893d-735bc95d6823 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:06.938652 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2d67a-53df-4c4b-893d-735bc95d6823 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:06.939416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2d67a-53df-4c4b-893d-735bc95d6823 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:06.939751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2d67a-53df-4c4b-893d-735bc95d6823 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:06.941199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2d67a-53df-4c4b-893d-735bc95d6823 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:06.952702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 406/816] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:06.953459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2d67a-53df-4c4b-893d-735bc95d6823 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:06.978598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:06.988639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aed60b-1a9b-4509-a75c-d17b196caeee None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.024756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fd7d5-405d-4966-b9f9-b37eef5d8a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.025435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd7d5-405d-4966-b9f9-b37eef5d8a1d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.025706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd7d5-405d-4966-b9f9-b37eef5d8a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.025975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd7d5-405d-4966-b9f9-b37eef5d8a1d None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.026691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd7d5-405d-4966-b9f9-b37eef5d8a1d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:07.026995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd7d5-405d-4966-b9f9-b37eef5d8a1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.028382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd7d5-405d-4966-b9f9-b37eef5d8a1d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.032897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 411/817] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 13 20:29:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:07.033676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fd7d5-405d-4966-b9f9-b37eef5d8a1d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:07.049115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2d67a-53df-4c4b-893d-735bc95d6823 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.076044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:07.084569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18390a8d-b981-4e5e-94ee-b420af03b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.085473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18390a8d-b981-4e5e-94ee-b420af03b4af None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.085759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18390a8d-b981-4e5e-94ee-b420af03b4af None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.086040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18390a8d-b981-4e5e-94ee-b420af03b4af None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/95dcfd3d412a47298d7d5cf7b3b56f97/roles/ee6ccebaaadd458b9d6b4397f2355b3c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.100146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18390a8d-b981-4e5e-94ee-b420af03b4af None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=95dcfd3d412a47298d7d5cf7b3b56f97, role_id=ee6ccebaaadd458b9d6b4397f2355b3c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.101609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18390a8d-b981-4e5e-94ee-b420af03b4af None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.115835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 407/818] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:29:06 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/95dcfd3d412a47298d7d5cf7b3b56f97/roles/ee6ccebaaadd458b9d6b4397f2355b3c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:29:07.116827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18390a8d-b981-4e5e-94ee-b420af03b4af None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:07.152949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fd7d5-405d-4966-b9f9-b37eef5d8a1d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.190518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3798a67-c9dc-4b11-8e57-6eda89ec9236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.191334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3798a67-c9dc-4b11-8e57-6eda89ec9236 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.191659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3798a67-c9dc-4b11-8e57-6eda89ec9236 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.191932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3798a67-c9dc-4b11-8e57-6eda89ec9236 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.200841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3798a67-c9dc-4b11-8e57-6eda89ec9236 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.202286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3798a67-c9dc-4b11-8e57-6eda89ec9236 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d3798a67-c9dc-4b11-8e57-6eda89ec9236 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:07.217685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:07.218742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 412/819] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 13 20:29:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:07.219745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3798a67-c9dc-4b11-8e57-6eda89ec9236 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:07.231194 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18390a8d-b981-4e5e-94ee-b420af03b4af None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.241788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:07.272987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e6c90-72e9-4bf8-bf20-dd207e5fac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.273782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e6c90-72e9-4bf8-bf20-dd207e5fac9d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.274073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e6c90-72e9-4bf8-bf20-dd207e5fac9d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.274351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e6c90-72e9-4bf8-bf20-dd207e5fac9d None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.275591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e6c90-72e9-4bf8-bf20-dd207e5fac9d None admin] RBAC: Adding query filter params (name=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:07.275591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e6c90-72e9-4bf8-bf20-dd207e5fac9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.276914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e6c90-72e9-4bf8-bf20-dd207e5fac9d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.284705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 408/820] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:07.284705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e6c90-72e9-4bf8-bf20-dd207e5fac9d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:07.329131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:07.343292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3798a67-c9dc-4b11-8e57-6eda89ec9236 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.379827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739408c-b0de-4ea0-905c-9c6383e8a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.381066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739408c-b0de-4ea0-905c-9c6383e8a773 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.381357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739408c-b0de-4ea0-905c-9c6383e8a773 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.381637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739408c-b0de-4ea0-905c-9c6383e8a773 None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/3f27793f58c74ffb9a7a8e4cdddb8169/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.383656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739408c-b0de-4ea0-905c-9c6383e8a773 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=3f27793f58c74ffb9a7a8e4cdddb8169, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.385104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739408c-b0de-4ea0-905c-9c6383e8a773 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.397297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e6c90-72e9-4bf8-bf20-dd207e5fac9d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.400417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 413/821] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:29:07 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/3f27793f58c74ffb9a7a8e4cdddb8169/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:29:07.400820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739408c-b0de-4ea0-905c-9c6383e8a773 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:07.455834 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e38be-b67f-4591-9e94-91fcb6201e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.457059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e38be-b67f-4591-9e94-91fcb6201e87 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.457488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e38be-b67f-4591-9e94-91fcb6201e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.458177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e38be-b67f-4591-9e94-91fcb6201e87 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.459551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e38be-b67f-4591-9e94-91fcb6201e87 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:07.460043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e38be-b67f-4591-9e94-91fcb6201e87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.461621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e38be-b67f-4591-9e94-91fcb6201e87 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.470400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 409/822] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 13 20:29:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:07.471896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e38be-b67f-4591-9e94-91fcb6201e87 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:07.491875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:07.529042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739408c-b0de-4ea0-905c-9c6383e8a773 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.575323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de54299-85ce-4fdb-b15c-dd5b6c37c3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.576268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de54299-85ce-4fdb-b15c-dd5b6c37c3db None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.576702 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de54299-85ce-4fdb-b15c-dd5b6c37c3db None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.576977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de54299-85ce-4fdb-b15c-dd5b6c37c3db None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.577816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de54299-85ce-4fdb-b15c-dd5b6c37c3db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:07.578125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de54299-85ce-4fdb-b15c-dd5b6c37c3db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.580150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de54299-85ce-4fdb-b15c-dd5b6c37c3db None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.587030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e38be-b67f-4591-9e94-91fcb6201e87 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.595339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:07.601925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 414/823] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 13 20:29:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:07.602759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de54299-85ce-4fdb-b15c-dd5b6c37c3db None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:07.632123 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b813b7-1872-4eed-9b51-63c847dfb74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.632941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b813b7-1872-4eed-9b51-63c847dfb74f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.633328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b813b7-1872-4eed-9b51-63c847dfb74f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.633749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b813b7-1872-4eed-9b51-63c847dfb74f None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.634574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b813b7-1872-4eed-9b51-63c847dfb74f None admin] RBAC: Adding query filter params (type=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:07.635011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b813b7-1872-4eed-9b51-63c847dfb74f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.636603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b813b7-1872-4eed-9b51-63c847dfb74f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.642333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 410/824] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:07.643503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b813b7-1872-4eed-9b51-63c847dfb74f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:07.707861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de54299-85ce-4fdb-b15c-dd5b6c37c3db None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.738569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:07.742381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091bfa7-448f-4f39-899f-0541773bbead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.743059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091bfa7-448f-4f39-899f-0541773bbead None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.743347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091bfa7-448f-4f39-899f-0541773bbead None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.743678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091bfa7-448f-4f39-899f-0541773bbead None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.744460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091bfa7-448f-4f39-899f-0541773bbead None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.745950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091bfa7-448f-4f39-899f-0541773bbead None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.748271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b813b7-1872-4eed-9b51-63c847dfb74f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.769601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 415/825] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 13 20:29:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:07.770554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091bfa7-448f-4f39-899f-0541773bbead None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:07.795953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4511fb3-3ed0-42aa-ad53-485fac7ca7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.797108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4511fb3-3ed0-42aa-ad53-485fac7ca7ba None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.797494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4511fb3-3ed0-42aa-ad53-485fac7ca7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.797878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4511fb3-3ed0-42aa-ad53-485fac7ca7ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.846688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:07.887645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091bfa7-448f-4f39-899f-0541773bbead None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:07.890127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4511fb3-3ed0-42aa-ad53-485fac7ca7ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.891958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4511fb3-3ed0-42aa-ad53-485fac7ca7ba None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.923071 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62792e2-b681-469c-86a4-b2ba4f2cc865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:07.923782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62792e2-b681-469c-86a4-b2ba4f2cc865 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:07.924065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62792e2-b681-469c-86a4-b2ba4f2cc865 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:07.924339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62792e2-b681-469c-86a4-b2ba4f2cc865 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:07.925084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62792e2-b681-469c-86a4-b2ba4f2cc865 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:07.926466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62792e2-b681-469c-86a4-b2ba4f2cc865 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:07.977489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 416/826] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 13 20:29:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:08.010983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 411/827] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:08.020779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62792e2-b681-469c-86a4-b2ba4f2cc865 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:08.075895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:08.142990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62792e2-b681-469c-86a4-b2ba4f2cc865 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:08.179569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669e84f-0f6c-4a69-9ddb-8072106e5dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:08.180384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669e84f-0f6c-4a69-9ddb-8072106e5dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:08.180672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669e84f-0f6c-4a69-9ddb-8072106e5dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:08.180952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669e84f-0f6c-4a69-9ddb-8072106e5dd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:08.192606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669e84f-0f6c-4a69-9ddb-8072106e5dd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:08.194102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669e84f-0f6c-4a69-9ddb-8072106e5dd9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1669e84f-0f6c-4a69-9ddb-8072106e5dd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:08.213454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:08.214779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:08.215710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:08.215710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:08.215710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:08.215710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:08.215710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 417/828] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:08.220338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4511fb3-3ed0-42aa-ad53-485fac7ca7ba None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:08.310021 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:08.348292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4511fb3-3ed0-42aa-ad53-485fac7ca7ba None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:08.384164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc3f6d-6094-410d-9a3d-eb0fef823ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:08.384874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc3f6d-6094-410d-9a3d-eb0fef823ecf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:08.385129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc3f6d-6094-410d-9a3d-eb0fef823ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:08.385414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc3f6d-6094-410d-9a3d-eb0fef823ecf None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:08.386217 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc3f6d-6094-410d-9a3d-eb0fef823ecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:08.386530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc3f6d-6094-410d-9a3d-eb0fef823ecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:08.387969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc3f6d-6094-410d-9a3d-eb0fef823ecf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:08.400816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 412/829] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:08.405831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669e84f-0f6c-4a69-9ddb-8072106e5dd9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:08.498937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:08.507641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669e84f-0f6c-4a69-9ddb-8072106e5dd9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:08.542419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11c701-efbb-4f5f-98e7-316de573ca78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:08.543607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11c701-efbb-4f5f-98e7-316de573ca78 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:08.544093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11c701-efbb-4f5f-98e7-316de573ca78 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:08.544566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11c701-efbb-4f5f-98e7-316de573ca78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:08.551911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11c701-efbb-4f5f-98e7-316de573ca78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:08.553610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11c701-efbb-4f5f-98e7-316de573ca78 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3c11c701-efbb-4f5f-98e7-316de573ca78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:08.568027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:08.568967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:08.569878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:08.569878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:08.569878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:08.569878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:08.569878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:08.569878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:08.569878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:08.569878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:08.572307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 418/830] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:08.578514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc3f6d-6094-410d-9a3d-eb0fef823ecf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:08.679156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc3f6d-6094-410d-9a3d-eb0fef823ecf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:08.708820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:08.717649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db01a9-41ec-49ba-bedd-794bca91add8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:08.718327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db01a9-41ec-49ba-bedd-794bca91add8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:08.718617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db01a9-41ec-49ba-bedd-794bca91add8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:08.718901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db01a9-41ec-49ba-bedd-794bca91add8 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:08.719735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db01a9-41ec-49ba-bedd-794bca91add8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:08.720270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db01a9-41ec-49ba-bedd-794bca91add8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:08.721905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db01a9-41ec-49ba-bedd-794bca91add8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:08.734540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 413/831] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:08.740380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11c701-efbb-4f5f-98e7-316de573ca78 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:08.841880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11c701-efbb-4f5f-98e7-316de573ca78 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:08.876856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422787e3-8824-4136-ae7b-98130b44fc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:08.877855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422787e3-8824-4136-ae7b-98130b44fc98 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:08.878362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422787e3-8824-4136-ae7b-98130b44fc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:08.878828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422787e3-8824-4136-ae7b-98130b44fc98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:08.879889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422787e3-8824-4136-ae7b-98130b44fc98 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=e80bb5b10ce945cabbfb88ca5df92e3c) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:08.880396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422787e3-8824-4136-ae7b-98130b44fc98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:08.882015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422787e3-8824-4136-ae7b-98130b44fc98 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:08.888585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 419/832] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:08 2022] GET /identity/v3/role_assignments?user.id=e80bb5b10ce945cabbfb88ca5df92e3c&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:09.237643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d0bdd-47bf-4749-a1c2-a1545731461c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:09.238028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d0bdd-47bf-4749-a1c2-a1545731461c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:09.238209 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d0bdd-47bf-4749-a1c2-a1545731461c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:09.239955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 414/833] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:09.245772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b975c-91d7-471f-b78d-d61d4813fd15 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:09.246286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b975c-91d7-471f-b78d-d61d4813fd15 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:09.246362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b975c-91d7-471f-b78d-d61d4813fd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:09.263447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-132b975c-91d7-471f-b78d-d61d4813fd15 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:09.264669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-132b975c-91d7-471f-b78d-d61d4813fd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:09.276063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:09.333485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 420/834] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:09.356437 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d0bdd-47bf-4749-a1c2-a1545731461c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:09.462922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d0bdd-47bf-4749-a1c2-a1545731461c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:09.497367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d934f-1ea4-42b9-8fa3-19baea352bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:09.498180 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d934f-1ea4-42b9-8fa3-19baea352bdc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:09.498469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d934f-1ea4-42b9-8fa3-19baea352bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:09.498753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d934f-1ea4-42b9-8fa3-19baea352bdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:09.509394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d934f-1ea4-42b9-8fa3-19baea352bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5c5d934f-1ea4-42b9-8fa3-19baea352bdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:09.517398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:09.518279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:09.519102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 415/835] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:29:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:09.524485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b975c-91d7-471f-b78d-d61d4813fd15 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:09.625741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b975c-91d7-471f-b78d-d61d4813fd15 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:09.645896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3b235-f591-436d-897e-4f2ec8d795af None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:09.646222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3b235-f591-436d-897e-4f2ec8d795af None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:09.646468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3b235-f591-436d-897e-4f2ec8d795af None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:09.649098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 416/836] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:09.653207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ba47e-ae68-4a14-84c6-dc2247155596 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:09.653511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ba47e-ae68-4a14-84c6-dc2247155596 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:09.653791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ba47e-ae68-4a14-84c6-dc2247155596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:09.660142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540d660-e488-4457-863d-1fe47079fcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:09.660848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540d660-e488-4457-863d-1fe47079fcab None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:09.661137 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540d660-e488-4457-863d-1fe47079fcab None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:09.661410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540d660-e488-4457-863d-1fe47079fcab None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:09.662158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540d660-e488-4457-863d-1fe47079fcab None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:09.662469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540d660-e488-4457-863d-1fe47079fcab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:09.663862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540d660-e488-4457-863d-1fe47079fcab None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:09.670807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6c7ba47e-ae68-4a14-84c6-dc2247155596 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:09.672049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6c7ba47e-ae68-4a14-84c6-dc2247155596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:09.672794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 421/837] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:29:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:09.677894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540d660-e488-4457-863d-1fe47079fcab None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:09.694991 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:09.731264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:09.749592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 417/838] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:09.774054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ba47e-ae68-4a14-84c6-dc2247155596 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:09.795762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540d660-e488-4457-863d-1fe47079fcab None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:09.830484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bb9b2-8d04-4daa-baa6-5163e3788aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:09.831297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb9b2-8d04-4daa-baa6-5163e3788aee None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:09.831652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb9b2-8d04-4daa-baa6-5163e3788aee None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:09.831935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb9b2-8d04-4daa-baa6-5163e3788aee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:09.843917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bb9b2-8d04-4daa-baa6-5163e3788aee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:09.845547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bb9b2-8d04-4daa-baa6-5163e3788aee None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-b17bb9b2-8d04-4daa-baa6-5163e3788aee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:09.863526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:09.864450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:09.865809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:09.865809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:09.865809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:09.865809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:09.865809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 422/839] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:09.870767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bb9b2-8d04-4daa-baa6-5163e3788aee None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:09.889260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ba47e-ae68-4a14-84c6-dc2247155596 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:09.924538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769bd3b-be14-44b4-a4d1-ef3b92c8c565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:09.925549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769bd3b-be14-44b4-a4d1-ef3b92c8c565 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:09.925927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769bd3b-be14-44b4-a4d1-ef3b92c8c565 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:09.926295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769bd3b-be14-44b4-a4d1-ef3b92c8c565 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:09.937084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769bd3b-be14-44b4-a4d1-ef3b92c8c565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:09.937550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f769bd3b-be14-44b4-a4d1-ef3b92c8c565 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:09.945022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:09.946156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:09.947210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:09.949642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 418/840] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:09.956288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769bd3b-be14-44b4-a4d1-ef3b92c8c565 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:09.977386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bb9b2-8d04-4daa-baa6-5163e3788aee None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.003949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:10.015411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935437a-e04f-47d5-b8e5-869f9081da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.016422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935437a-e04f-47d5-b8e5-869f9081da05 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.016853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935437a-e04f-47d5-b8e5-869f9081da05 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.017269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935437a-e04f-47d5-b8e5-869f9081da05 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.018260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935437a-e04f-47d5-b8e5-869f9081da05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:10.018712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935437a-e04f-47d5-b8e5-869f9081da05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.020357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935437a-e04f-47d5-b8e5-869f9081da05 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.034159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 423/841] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:10.039527 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935437a-e04f-47d5-b8e5-869f9081da05 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.088639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769bd3b-be14-44b4-a4d1-ef3b92c8c565 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.125280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432fd36-80d7-4e63-850a-0502da4174ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.125953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432fd36-80d7-4e63-850a-0502da4174ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.126235 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432fd36-80d7-4e63-850a-0502da4174ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.126563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432fd36-80d7-4e63-850a-0502da4174ac None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.127309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432fd36-80d7-4e63-850a-0502da4174ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:10.127690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432fd36-80d7-4e63-850a-0502da4174ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.129076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432fd36-80d7-4e63-850a-0502da4174ac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.138264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 419/842] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:29:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:10.143593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432fd36-80d7-4e63-850a-0502da4174ac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.154756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935437a-e04f-47d5-b8e5-869f9081da05 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.175678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:10.191282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28f160-dfbd-4559-ba69-0e490ae03b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.192178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28f160-dfbd-4559-ba69-0e490ae03b9b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.192623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28f160-dfbd-4559-ba69-0e490ae03b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.192905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28f160-dfbd-4559-ba69-0e490ae03b9b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.202893 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28f160-dfbd-4559-ba69-0e490ae03b9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.204982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28f160-dfbd-4559-ba69-0e490ae03b9b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-6f28f160-dfbd-4559-ba69-0e490ae03b9b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:10.221035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:10.221993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:10.222930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:10.226713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 424/843] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:10.228027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03262174-6561-45d5-8f78-9e7b2b91387b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.228318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03262174-6561-45d5-8f78-9e7b2b91387b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.228598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03262174-6561-45d5-8f78-9e7b2b91387b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.230042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 425/844] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:10.230621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03262174-6561-45d5-8f78-9e7b2b91387b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.247079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:10.251309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432fd36-80d7-4e63-850a-0502da4174ac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.288447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43d869-c0b5-4052-9d0a-18329f383b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.289247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43d869-c0b5-4052-9d0a-18329f383b82 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.289535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43d869-c0b5-4052-9d0a-18329f383b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.289813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43d869-c0b5-4052-9d0a-18329f383b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.300831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43d869-c0b5-4052-9d0a-18329f383b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.302431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43d869-c0b5-4052-9d0a-18329f383b82 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4b43d869-c0b5-4052-9d0a-18329f383b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:10.319175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:10.320078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:10.320955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:10.320955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:10.320955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:10.320955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:10.320955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 420/845] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:10.321640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545e320-9b77-4313-8a25-6c63f9fbe37f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.321813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545e320-9b77-4313-8a25-6c63f9fbe37f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.322096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545e320-9b77-4313-8a25-6c63f9fbe37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.338163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e545e320-9b77-4313-8a25-6c63f9fbe37f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:10.339404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e545e320-9b77-4313-8a25-6c63f9fbe37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:10.345496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03262174-6561-45d5-8f78-9e7b2b91387b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.382205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3783e-ecc6-4c23-9427-65951621b3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.382885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3783e-ecc6-4c23-9427-65951621b3eb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.383173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3783e-ecc6-4c23-9427-65951621b3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.383474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3783e-ecc6-4c23-9427-65951621b3eb None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.384323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3783e-ecc6-4c23-9427-65951621b3eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:10.384638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3783e-ecc6-4c23-9427-65951621b3eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.386120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3783e-ecc6-4c23-9427-65951621b3eb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.408682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 426/846] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:10.408682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3783e-ecc6-4c23-9427-65951621b3eb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.413323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 421/847] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:10.424685 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545e320-9b77-4313-8a25-6c63f9fbe37f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.449822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:10.511326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:10.512888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3783e-ecc6-4c23-9427-65951621b3eb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.534089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545e320-9b77-4313-8a25-6c63f9fbe37f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.550881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a81d5-1174-4eba-b089-5d0e53e4f7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.551663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a81d5-1174-4eba-b089-5d0e53e4f7ea None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.551948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a81d5-1174-4eba-b089-5d0e53e4f7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.552222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a81d5-1174-4eba-b089-5d0e53e4f7ea None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.552935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a81d5-1174-4eba-b089-5d0e53e4f7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:10.553240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a81d5-1174-4eba-b089-5d0e53e4f7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.554714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a81d5-1174-4eba-b089-5d0e53e4f7ea None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.571378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 427/848] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:10.571574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f5da5-187c-4bf6-ab66-e8a3d0bd8ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.571574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f5da5-187c-4bf6-ab66-e8a3d0bd8ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.571795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a81d5-1174-4eba-b089-5d0e53e4f7ea None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.571881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f5da5-187c-4bf6-ab66-e8a3d0bd8ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.571881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f5da5-187c-4bf6-ab66-e8a3d0bd8ea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.637736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f5da5-187c-4bf6-ab66-e8a3d0bd8ea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.639209 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f5da5-187c-4bf6-ab66-e8a3d0bd8ea5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.671180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:10.676061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a81d5-1174-4eba-b089-5d0e53e4f7ea None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.711672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f2c1a-c8bd-417e-9af8-74f1dc96ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.712480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f2c1a-c8bd-417e-9af8-74f1dc96ec09 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.712760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f2c1a-c8bd-417e-9af8-74f1dc96ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.713035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f2c1a-c8bd-417e-9af8-74f1dc96ec09 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.713817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f2c1a-c8bd-417e-9af8-74f1dc96ec09 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.715179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f2c1a-c8bd-417e-9af8-74f1dc96ec09 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.719782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fd3f2c1a-c8bd-417e-9af8-74f1dc96ec09 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:10.727340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:10.728366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:10.729205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 428/849] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:10.730019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f2c1a-c8bd-417e-9af8-74f1dc96ec09 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.753580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 422/850] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:10.754530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f5da5-187c-4bf6-ab66-e8a3d0bd8ea5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.835172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f2c1a-c8bd-417e-9af8-74f1dc96ec09 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.857109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f5da5-187c-4bf6-ab66-e8a3d0bd8ea5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:10.869829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e9fff-5352-4e4f-bba0-0cfba04fb6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.870663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e9fff-5352-4e4f-bba0-0cfba04fb6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.870941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e9fff-5352-4e4f-bba0-0cfba04fb6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.871211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e9fff-5352-4e4f-bba0-0cfba04fb6b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.878989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e9fff-5352-4e4f-bba0-0cfba04fb6b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.880446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e9fff-5352-4e4f-bba0-0cfba04fb6b0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fa4e9fff-5352-4e4f-bba0-0cfba04fb6b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:10.894401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:10.895309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:10.896287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 429/851] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:10.896858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e9fff-5352-4e4f-bba0-0cfba04fb6b0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.904972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00ac7c-6d98-443d-8b85-2cb418441405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:10.905658 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00ac7c-6d98-443d-8b85-2cb418441405 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:10.906283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00ac7c-6d98-443d-8b85-2cb418441405 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:10.906283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00ac7c-6d98-443d-8b85-2cb418441405 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:10.906933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00ac7c-6d98-443d-8b85-2cb418441405 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:10.907249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00ac7c-6d98-443d-8b85-2cb418441405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:10.908681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00ac7c-6d98-443d-8b85-2cb418441405 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:10.914067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 423/852] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:10.914725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00ac7c-6d98-443d-8b85-2cb418441405 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:10.925722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:10.964396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:11.012657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e9fff-5352-4e4f-bba0-0cfba04fb6b0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.021953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00ac7c-6d98-443d-8b85-2cb418441405 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.046737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb09fc-64be-4d33-af09-bdc657e08c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.047582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb09fc-64be-4d33-af09-bdc657e08c68 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.047870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb09fc-64be-4d33-af09-bdc657e08c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.051060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb09fc-64be-4d33-af09-bdc657e08c68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.057230 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11f0aa-edcf-4b38-9618-3ee897064b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.058044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11f0aa-edcf-4b38-9618-3ee897064b80 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.058333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11f0aa-edcf-4b38-9618-3ee897064b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.058614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11f0aa-edcf-4b38-9618-3ee897064b80 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.059473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11f0aa-edcf-4b38-9618-3ee897064b80 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:11.059793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11f0aa-edcf-4b38-9618-3ee897064b80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.061251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11f0aa-edcf-4b38-9618-3ee897064b80 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.063422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb09fc-64be-4d33-af09-bdc657e08c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.065649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb09fc-64be-4d33-af09-bdc657e08c68 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.079426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 424/853] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:11.080152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11f0aa-edcf-4b38-9618-3ee897064b80 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d9cb09fc-64be-4d33-af09-bdc657e08c68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:11.082881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:11.083816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:11.084674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:11.084674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:11.084674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:11.084674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:11.084674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 430/854] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:11.085506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253a7f7-8879-4b62-ba54-e6d4b1aa0abe None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.085811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253a7f7-8879-4b62-ba54-e6d4b1aa0abe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.086097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253a7f7-8879-4b62-ba54-e6d4b1aa0abe None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.090032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 431/855] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:11.090032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253a7f7-8879-4b62-ba54-e6d4b1aa0abe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.164681 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:11.179061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11f0aa-edcf-4b38-9618-3ee897064b80 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.180558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:11.189621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253a7f7-8879-4b62-ba54-e6d4b1aa0abe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.213780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea02e81-a81a-4c42-aad1-3a0c80bf5a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.214454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea02e81-a81a-4c42-aad1-3a0c80bf5a46 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.214737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea02e81-a81a-4c42-aad1-3a0c80bf5a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.215010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea02e81-a81a-4c42-aad1-3a0c80bf5a46 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.215759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea02e81-a81a-4c42-aad1-3a0c80bf5a46 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:11.216072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea02e81-a81a-4c42-aad1-3a0c80bf5a46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.217463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea02e81-a81a-4c42-aad1-3a0c80bf5a46 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.222164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 425/856] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:11.222458 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa34361-c3d6-4e1f-91f2-79815444783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.222946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea02e81-a81a-4c42-aad1-3a0c80bf5a46 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.223101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa34361-c3d6-4e1f-91f2-79815444783a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.223483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa34361-c3d6-4e1f-91f2-79815444783a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.223769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa34361-c3d6-4e1f-91f2-79815444783a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.307698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa34361-c3d6-4e1f-91f2-79815444783a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.309193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa34361-c3d6-4e1f-91f2-79815444783a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.341201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea02e81-a81a-4c42-aad1-3a0c80bf5a46 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.382559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c398286-0303-42ab-ac18-c45d1ecc62ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.383232 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c398286-0303-42ab-ac18-c45d1ecc62ae None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.383552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c398286-0303-42ab-ac18-c45d1ecc62ae None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.383831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c398286-0303-42ab-ac18-c45d1ecc62ae None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.384555 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c398286-0303-42ab-ac18-c45d1ecc62ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:11.384863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c398286-0303-42ab-ac18-c45d1ecc62ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.386562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c398286-0303-42ab-ac18-c45d1ecc62ae None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.392157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:11.400909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 426/857] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:11.402332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41add2-ff30-4112-b59f-6b4359157140 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.402624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41add2-ff30-4112-b59f-6b4359157140 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.402906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41add2-ff30-4112-b59f-6b4359157140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.420241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0e41add2-ff30-4112-b59f-6b4359157140 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:11.421421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0e41add2-ff30-4112-b59f-6b4359157140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:11.424787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 432/858] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:11.425362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa34361-c3d6-4e1f-91f2-79815444783a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.445362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:11.494157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 427/859] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:11.494815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41add2-ff30-4112-b59f-6b4359157140 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.542112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa34361-c3d6-4e1f-91f2-79815444783a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.582329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ccd5c3-3bf3-4e36-89ed-aa7553d2f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.583650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ccd5c3-3bf3-4e36-89ed-aa7553d2f444 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.583936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ccd5c3-3bf3-4e36-89ed-aa7553d2f444 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.584355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ccd5c3-3bf3-4e36-89ed-aa7553d2f444 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.591218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ccd5c3-3bf3-4e36-89ed-aa7553d2f444 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.592912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ccd5c3-3bf3-4e36-89ed-aa7553d2f444 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.595732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41add2-ff30-4112-b59f-6b4359157140 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c4ccd5c3-3bf3-4e36-89ed-aa7553d2f444 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:11.606322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:11.607189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 433/860] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:11.608845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ccd5c3-3bf3-4e36-89ed-aa7553d2f444 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.635971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5d8fe-55eb-4526-bb61-638e2a4a13c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.636782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5d8fe-55eb-4526-bb61-638e2a4a13c9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.637056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5d8fe-55eb-4526-bb61-638e2a4a13c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.637337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5d8fe-55eb-4526-bb61-638e2a4a13c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.646267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:11.648711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5d8fe-55eb-4526-bb61-638e2a4a13c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.650210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5d8fe-55eb-4526-bb61-638e2a4a13c9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4cb5d8fe-55eb-4526-bb61-638e2a4a13c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:11.668485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:11.669405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:11.670377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:11.670377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:11.670377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:11.670377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:11.674491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 428/861] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:11.675244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5d8fe-55eb-4526-bb61-638e2a4a13c9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.687416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:11.719359 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ccd5c3-3bf3-4e36-89ed-aa7553d2f444 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.757960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e88ab-6f8c-4669-bc47-94db3564b963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.758787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e88ab-6f8c-4669-bc47-94db3564b963 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.759072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e88ab-6f8c-4669-bc47-94db3564b963 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.759349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e88ab-6f8c-4669-bc47-94db3564b963 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.760078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e88ab-6f8c-4669-bc47-94db3564b963 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.761899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e88ab-6f8c-4669-bc47-94db3564b963 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-340e88ab-6f8c-4669-bc47-94db3564b963 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:11.778336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:11.779293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:29:11.780516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:11.781811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 434/862] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:11.782635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e88ab-6f8c-4669-bc47-94db3564b963 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.787780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5d8fe-55eb-4526-bb61-638e2a4a13c9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.823475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be55f0d-0cab-42b6-816c-9609bc7bffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.824285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be55f0d-0cab-42b6-816c-9609bc7bffb7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.824575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be55f0d-0cab-42b6-816c-9609bc7bffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.824854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be55f0d-0cab-42b6-816c-9609bc7bffb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.825908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be55f0d-0cab-42b6-816c-9609bc7bffb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:11.826015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be55f0d-0cab-42b6-816c-9609bc7bffb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.827393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be55f0d-0cab-42b6-816c-9609bc7bffb7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.844795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 429/863] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:11.844795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be55f0d-0cab-42b6-816c-9609bc7bffb7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.886706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:11.891145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e88ab-6f8c-4669-bc47-94db3564b963 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.918403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:11.928629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8db49-86e8-4a5e-83ee-e47919d3668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.929305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8db49-86e8-4a5e-83ee-e47919d3668e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.929581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8db49-86e8-4a5e-83ee-e47919d3668e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.929857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8db49-86e8-4a5e-83ee-e47919d3668e None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.930579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8db49-86e8-4a5e-83ee-e47919d3668e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:11.931065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8db49-86e8-4a5e-83ee-e47919d3668e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.932305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8db49-86e8-4a5e-83ee-e47919d3668e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:11.944422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 435/864] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:11.950014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8db49-86e8-4a5e-83ee-e47919d3668e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:11.956988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be55f0d-0cab-42b6-816c-9609bc7bffb7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:11.991802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801ced3-a5bf-4228-bcb8-cf646707112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:11.992405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ced3-a5bf-4228-bcb8-cf646707112c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:11.992698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ced3-a5bf-4228-bcb8-cf646707112c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:11.992979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ced3-a5bf-4228-bcb8-cf646707112c None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:11.993682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801ced3-a5bf-4228-bcb8-cf646707112c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:11.993997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801ced3-a5bf-4228-bcb8-cf646707112c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:11.995417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801ced3-a5bf-4228-bcb8-cf646707112c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:12.006610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 430/865] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:12.006610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801ced3-a5bf-4228-bcb8-cf646707112c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:12.069503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8db49-86e8-4a5e-83ee-e47919d3668e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:12.126600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801ced3-a5bf-4228-bcb8-cf646707112c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:12.145786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f072c0-662a-475b-9ff3-14aede6afcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:12.146451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f072c0-662a-475b-9ff3-14aede6afcef None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:12.146737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f072c0-662a-475b-9ff3-14aede6afcef None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:12.147017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f072c0-662a-475b-9ff3-14aede6afcef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:12.147894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f072c0-662a-475b-9ff3-14aede6afcef None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=95dcfd3d412a47298d7d5cf7b3b56f97) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:12.148212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f072c0-662a-475b-9ff3-14aede6afcef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:12.149659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f072c0-662a-475b-9ff3-14aede6afcef None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:12.165277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 436/866] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:11 2022] GET /identity/v3/role_assignments?user.id=95dcfd3d412a47298d7d5cf7b3b56f97&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:12.165277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f072c0-662a-475b-9ff3-14aede6afcef None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:12.166419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:12.172356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110f410-41cc-417c-b2ff-579345bb97b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:12.174741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110f410-41cc-417c-b2ff-579345bb97b3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:12.177171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110f410-41cc-417c-b2ff-579345bb97b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:12.177676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110f410-41cc-417c-b2ff-579345bb97b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:12.195055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110f410-41cc-417c-b2ff-579345bb97b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:12.200618 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110f410-41cc-417c-b2ff-579345bb97b3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9110f410-41cc-417c-b2ff-579345bb97b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:12.228620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:12.229815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:12.230688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:12.230688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:12.230688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:12.230688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:12.230688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:12.230688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:12.230688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:12.230688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:12.239901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:12.249619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 431/867] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:12.250813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110f410-41cc-417c-b2ff-579345bb97b3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:12.372770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f072c0-662a-475b-9ff3-14aede6afcef None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:12.407958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4a350-4128-4b86-9f25-994d26d244fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:12.408616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4a350-4128-4b86-9f25-994d26d244fb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:12.408908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4a350-4128-4b86-9f25-994d26d244fb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:12.409248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4a350-4128-4b86-9f25-994d26d244fb None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:12.409959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4a350-4128-4b86-9f25-994d26d244fb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:12.410313 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4a350-4128-4b86-9f25-994d26d244fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:12.411721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4a350-4128-4b86-9f25-994d26d244fb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:12.417304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 437/868] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:12.428668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110f410-41cc-417c-b2ff-579345bb97b3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:12.470271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaa247-f75e-4242-bf17-9e8f73c0d98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:12.471139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa247-f75e-4242-bf17-9e8f73c0d98b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:12.471551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa247-f75e-4242-bf17-9e8f73c0d98b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:12.471923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa247-f75e-4242-bf17-9e8f73c0d98b None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:12.472845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa247-f75e-4242-bf17-9e8f73c0d98b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:12.473328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa247-f75e-4242-bf17-9e8f73c0d98b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:12.475097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa247-f75e-4242-bf17-9e8f73c0d98b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:12.495634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 432/869] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:12.501290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4a350-4128-4b86-9f25-994d26d244fb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:12.601413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:12.617638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4a350-4128-4b86-9f25-994d26d244fb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:12.652392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059efc9-7800-4dc4-833b-0d8e2071ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:12.653064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059efc9-7800-4dc4-833b-0d8e2071ce8e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:12.653352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059efc9-7800-4dc4-833b-0d8e2071ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:12.653636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059efc9-7800-4dc4-833b-0d8e2071ce8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:12.654475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059efc9-7800-4dc4-833b-0d8e2071ce8e None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=3f27793f58c74ffb9a7a8e4cdddb8169) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:12.654791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059efc9-7800-4dc4-833b-0d8e2071ce8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:12.656253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059efc9-7800-4dc4-833b-0d8e2071ce8e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:12.667438 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 438/870] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:12 2022] GET /identity/v3/role_assignments?user.id=3f27793f58c74ffb9a7a8e4cdddb8169&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:13.226377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a8ca0-8659-4120-9548-f9594cebb8ea None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:13.226755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a8ca0-8659-4120-9548-f9594cebb8ea None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:13.226934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a8ca0-8659-4120-9548-f9594cebb8ea None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:13.228544 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 433/871] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:13.233763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4882-84b3-4bb1-8eb3-81398826f4e7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:13.234051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4882-84b3-4bb1-8eb3-81398826f4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:13.234330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4882-84b3-4bb1-8eb3-81398826f4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:13.251083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-015e4882-84b3-4bb1-8eb3-81398826f4e7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:13.252303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-015e4882-84b3-4bb1-8eb3-81398826f4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:13.316429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 439/872] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:13.338700 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a8ca0-8659-4120-9548-f9594cebb8ea None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:13.396513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:13.447012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a8ca0-8659-4120-9548-f9594cebb8ea None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:13.482141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb5c99-68d1-4a90-8ba0-140a7cc43eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:13.482836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb5c99-68d1-4a90-8ba0-140a7cc43eb7 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:13.483129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb5c99-68d1-4a90-8ba0-140a7cc43eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:13.483392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb5c99-68d1-4a90-8ba0-140a7cc43eb7 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:13.484187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb5c99-68d1-4a90-8ba0-140a7cc43eb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:13.485696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb5c99-68d1-4a90-8ba0-140a7cc43eb7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:13.506425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 434/873] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:29:13 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:14.404952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae579a8-0625-4687-a4d6-e05bebbd1bdc None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:14.405447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae579a8-0625-4687-a4d6-e05bebbd1bdc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:14.405447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae579a8-0625-4687-a4d6-e05bebbd1bdc None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:14.407070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 440/874] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:14.412291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ff634-327f-4c3c-b363-3b3512980c24 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:14.412851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ff634-327f-4c3c-b363-3b3512980c24 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:14.413333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ff634-327f-4c3c-b363-3b3512980c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:14.430634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f97ff634-327f-4c3c-b363-3b3512980c24 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:14.432236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f97ff634-327f-4c3c-b363-3b3512980c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:14.512740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 435/875] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:14 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:14.514513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760360d-ad78-4f20-ac69-351a56d80c87 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:14.514810 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760360d-ad78-4f20-ac69-351a56d80c87 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:14.515092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760360d-ad78-4f20-ac69-351a56d80c87 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:14.516474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 441/876] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:14.521637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3227b2d-4cb8-4d78-9ccf-5323fcff2d0c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:14.522147 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3227b2d-4cb8-4d78-9ccf-5323fcff2d0c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:14.522619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3227b2d-4cb8-4d78-9ccf-5323fcff2d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:14.536368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760360d-ad78-4f20-ac69-351a56d80c87 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:14.541499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c3227b2d-4cb8-4d78-9ccf-5323fcff2d0c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:14.542772 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c3227b2d-4cb8-4d78-9ccf-5323fcff2d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:14.578833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:14.623764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 436/877] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:14 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:14.629867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:14.642551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3227b2d-4cb8-4d78-9ccf-5323fcff2d0c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:14.651951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760360d-ad78-4f20-ac69-351a56d80c87 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:14.698841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2c0a2-ebc8-4953-8e8c-35f32cdbd984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:14.700210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2c0a2-ebc8-4953-8e8c-35f32cdbd984 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:14.700643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2c0a2-ebc8-4953-8e8c-35f32cdbd984 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:14.701062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2c0a2-ebc8-4953-8e8c-35f32cdbd984 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:14.712581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2c0a2-ebc8-4953-8e8c-35f32cdbd984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-b5b2c0a2-ebc8-4953-8e8c-35f32cdbd984 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:14.721992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:14.723141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:14.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:14.726609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 442/878] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 13 20:29:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:14.735048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2c0a2-ebc8-4953-8e8c-35f32cdbd984 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:14.742502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3227b2d-4cb8-4d78-9ccf-5323fcff2d0c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:14.777680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49dddb-4a14-4e0c-9c56-8539e1faa4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:14.778346 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49dddb-4a14-4e0c-9c56-8539e1faa4c1 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:14.778630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49dddb-4a14-4e0c-9c56-8539e1faa4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:14.778903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49dddb-4a14-4e0c-9c56-8539e1faa4c1 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:14.779835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49dddb-4a14-4e0c-9c56-8539e1faa4c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:14.781025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49dddb-4a14-4e0c-9c56-8539e1faa4c1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:14.796107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 437/879] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:29:14 2022] POST /identity/v3/services => generated 269 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:14.797395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391067-f9d8-429a-84a4-dcc152c3a76c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:14.797695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391067-f9d8-429a-84a4-dcc152c3a76c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:14.797965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391067-f9d8-429a-84a4-dcc152c3a76c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:14.799186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 438/880] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:14.809553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192527-e5bc-4869-af82-f09a31140b75 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:14.810131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192527-e5bc-4869-af82-f09a31140b75 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:14.810952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192527-e5bc-4869-af82-f09a31140b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:14.836478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ab192527-e5bc-4869-af82-f09a31140b75 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:14.838655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab192527-e5bc-4869-af82-f09a31140b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:14.851343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2c0a2-ebc8-4953-8e8c-35f32cdbd984 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:14.873722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:14.899319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a3273-83ca-43d1-9ed5-3bae18eb053e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:14.900395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a3273-83ca-43d1-9ed5-3bae18eb053e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:14.900826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a3273-83ca-43d1-9ed5-3bae18eb053e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:14.901346 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a3273-83ca-43d1-9ed5-3bae18eb053e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:14.902243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a3273-83ca-43d1-9ed5-3bae18eb053e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:14.902746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a3273-83ca-43d1-9ed5-3bae18eb053e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:14.904446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a3273-83ca-43d1-9ed5-3bae18eb053e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:14.907811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:14.917423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 443/881] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 13 20:29:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:14.922726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a3273-83ca-43d1-9ed5-3bae18eb053e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:14.938113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 439/882] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:14.960885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab192527-e5bc-4869-af82-f09a31140b75 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:15.057470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a3273-83ca-43d1-9ed5-3bae18eb053e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:15.076604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab192527-e5bc-4869-af82-f09a31140b75 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:15.107879 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2465da5-b8d6-466c-9721-86619b8dc11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:15.108782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2465da5-b8d6-466c-9721-86619b8dc11b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.109267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2465da5-b8d6-466c-9721-86619b8dc11b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.109580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2465da5-b8d6-466c-9721-86619b8dc11b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.115836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922434ef-f116-41e4-a15c-92d7ff4802ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:15.116842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922434ef-f116-41e4-a15c-92d7ff4802ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.117793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922434ef-f116-41e4-a15c-92d7ff4802ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.118223 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922434ef-f116-41e4-a15c-92d7ff4802ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.119028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922434ef-f116-41e4-a15c-92d7ff4802ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:15.121041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922434ef-f116-41e4-a15c-92d7ff4802ac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:15.123777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2465da5-b8d6-466c-9721-86619b8dc11b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:15.125334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2465da5-b8d6-466c-9721-86619b8dc11b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-922434ef-f116-41e4-a15c-92d7ff4802ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:15.139624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:15.140641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:15.141452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c2465da5-b8d6-466c-9721-86619b8dc11b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:15.147558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:15.148421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:15.149291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:15.149291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:15.149291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:15.149291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:15.149291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 444/884] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:15.149572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 440/884] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:15.152089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2465da5-b8d6-466c-9721-86619b8dc11b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:15.152945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922434ef-f116-41e4-a15c-92d7ff4802ac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:15.184382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:15.215382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:15.269352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2465da5-b8d6-466c-9721-86619b8dc11b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:15.273214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922434ef-f116-41e4-a15c-92d7ff4802ac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:15.306067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d1764-c816-4c5e-ade6-7ec520ee1cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:15.306750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d1764-c816-4c5e-ade6-7ec520ee1cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.307040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d1764-c816-4c5e-ade6-7ec520ee1cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.307351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d1764-c816-4c5e-ade6-7ec520ee1cf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.308129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d1764-c816-4c5e-ade6-7ec520ee1cf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:15.308439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d1764-c816-4c5e-ade6-7ec520ee1cf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:15.309843 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d1764-c816-4c5e-ade6-7ec520ee1cf0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:15.311307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b75fa9-33a4-4087-8ba6-4ebd9f95693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:15.312410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b75fa9-33a4-4087-8ba6-4ebd9f95693f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.312935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b75fa9-33a4-4087-8ba6-4ebd9f95693f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.313407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b75fa9-33a4-4087-8ba6-4ebd9f95693f None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.314326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b75fa9-33a4-4087-8ba6-4ebd9f95693f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:15.314837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b75fa9-33a4-4087-8ba6-4ebd9f95693f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:15.316469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b75fa9-33a4-4087-8ba6-4ebd9f95693f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:15.326817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 445/886] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:15.326986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 441/886] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:15.328354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d1764-c816-4c5e-ade6-7ec520ee1cf0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:15.332666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b75fa9-33a4-4087-8ba6-4ebd9f95693f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:15.424474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:15.440001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d1764-c816-4c5e-ade6-7ec520ee1cf0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:15.442663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b75fa9-33a4-4087-8ba6-4ebd9f95693f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:15.451534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:15.477307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b85cd5-17a9-42aa-a5f4-7aa4adfabd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:15.478058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b85cd5-17a9-42aa-a5f4-7aa4adfabd7c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.478348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b85cd5-17a9-42aa-a5f4-7aa4adfabd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.478667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b85cd5-17a9-42aa-a5f4-7aa4adfabd7c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.482457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dcb27-9bd2-430e-9f96-d301b8bb77d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:15.483357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dcb27-9bd2-430e-9f96-d301b8bb77d0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.483860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dcb27-9bd2-430e-9f96-d301b8bb77d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.484363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dcb27-9bd2-430e-9f96-d301b8bb77d0 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.485273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dcb27-9bd2-430e-9f96-d301b8bb77d0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:15.485776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dcb27-9bd2-430e-9f96-d301b8bb77d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:15.486459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b85cd5-17a9-42aa-a5f4-7aa4adfabd7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:15.487584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dcb27-9bd2-430e-9f96-d301b8bb77d0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:15.487948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b85cd5-17a9-42aa-a5f4-7aa4adfabd7c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:15.499604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 442/887] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a6b85cd5-17a9-42aa-a5f4-7aa4adfabd7c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:15.503600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:15.504580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:15.505716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 446/888] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:29:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:15.510876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dcb27-9bd2-430e-9f96-d301b8bb77d0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:15.561404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacc117-eecb-4319-a508-ddd9638af6c7 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.562032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacc117-eecb-4319-a508-ddd9638af6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.562605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacc117-eecb-4319-a508-ddd9638af6c7 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.564607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 447/889] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:15.569895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4f101-9810-4209-a048-0b5083405395 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.570223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4f101-9810-4209-a048-0b5083405395 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.570446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4f101-9810-4209-a048-0b5083405395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.589295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dcf4f101-9810-4209-a048-0b5083405395 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:15.590530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dcf4f101-9810-4209-a048-0b5083405395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:15.623587 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dcb27-9bd2-430e-9f96-d301b8bb77d0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:15.662422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ded8c-4e92-4a4d-b9a4-9a0a091aa725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:15.663106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ded8c-4e92-4a4d-b9a4-9a0a091aa725 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.663400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ded8c-4e92-4a4d-b9a4-9a0a091aa725 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.663708 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ded8c-4e92-4a4d-b9a4-9a0a091aa725 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.665371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ded8c-4e92-4a4d-b9a4-9a0a091aa725 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:15.665689 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ded8c-4e92-4a4d-b9a4-9a0a091aa725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:15.667187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ded8c-4e92-4a4d-b9a4-9a0a091aa725 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:15.667530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 448/890] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:15.685282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4f101-9810-4209-a048-0b5083405395 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:15.686958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 443/891] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 13 20:29:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:15.692761 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ded8c-4e92-4a4d-b9a4-9a0a091aa725 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:15.711860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:15.797519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:15.801451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4f101-9810-4209-a048-0b5083405395 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:15.805901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ded8c-4e92-4a4d-b9a4-9a0a091aa725 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:15.840942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a1429-b1f8-4056-8a5e-64d594abcf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:15.841594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a1429-b1f8-4056-8a5e-64d594abcf62 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.841878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a1429-b1f8-4056-8a5e-64d594abcf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.842154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a1429-b1f8-4056-8a5e-64d594abcf62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.842817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73a576-40fa-4bfd-a8e8-e08e5cdc9cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:15.843904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73a576-40fa-4bfd-a8e8-e08e5cdc9cbc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:15.844384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73a576-40fa-4bfd-a8e8-e08e5cdc9cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:15.844851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73a576-40fa-4bfd-a8e8-e08e5cdc9cbc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:15.845711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73a576-40fa-4bfd-a8e8-e08e5cdc9cbc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:15.847283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73a576-40fa-4bfd-a8e8-e08e5cdc9cbc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-af73a576-40fa-4bfd-a8e8-e08e5cdc9cbc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:15.860189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:15.861188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:29:15.862009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:15.864907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 449/892] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:29:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:15.870854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73a576-40fa-4bfd-a8e8-e08e5cdc9cbc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:15.928127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a1429-b1f8-4056-8a5e-64d594abcf62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:15.929842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a1429-b1f8-4056-8a5e-64d594abcf62 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:15.942843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:15.998878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73a576-40fa-4bfd-a8e8-e08e5cdc9cbc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:16.037815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bdae3-36d1-4378-916f-9ed8fee01147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:16.038514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bdae3-36d1-4378-916f-9ed8fee01147 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.038795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bdae3-36d1-4378-916f-9ed8fee01147 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.039180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bdae3-36d1-4378-916f-9ed8fee01147 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.039941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bdae3-36d1-4378-916f-9ed8fee01147 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:16.040251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bdae3-36d1-4378-916f-9ed8fee01147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:16.041783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bdae3-36d1-4378-916f-9ed8fee01147 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:16.047494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 450/893] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:29:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:16.054229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bdae3-36d1-4378-916f-9ed8fee01147 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:16.068738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 444/894] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:15 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:29:16.074750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a1429-b1f8-4056-8a5e-64d594abcf62 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:16.091035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:16.169235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bdae3-36d1-4378-916f-9ed8fee01147 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:16.194815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a1429-b1f8-4056-8a5e-64d594abcf62 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:16.206194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd542d5-3cc7-40fd-8bf4-ac2cd8777cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:16.206680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:16.208032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd542d5-3cc7-40fd-8bf4-ac2cd8777cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.208567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd542d5-3cc7-40fd-8bf4-ac2cd8777cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.209145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd542d5-3cc7-40fd-8bf4-ac2cd8777cd2 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.210064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd542d5-3cc7-40fd-8bf4-ac2cd8777cd2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:16.210567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd542d5-3cc7-40fd-8bf4-ac2cd8777cd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:16.212235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd542d5-3cc7-40fd-8bf4-ac2cd8777cd2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:16.218126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 451/895] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:29:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:16.224242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd542d5-3cc7-40fd-8bf4-ac2cd8777cd2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:16.239841 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80433f6-45e7-43e8-9891-0ef04840c84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:16.240981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80433f6-45e7-43e8-9891-0ef04840c84b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.241489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80433f6-45e7-43e8-9891-0ef04840c84b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.241961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80433f6-45e7-43e8-9891-0ef04840c84b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.253456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80433f6-45e7-43e8-9891-0ef04840c84b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:16.255308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80433f6-45e7-43e8-9891-0ef04840c84b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f80433f6-45e7-43e8-9891-0ef04840c84b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:16.274333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:16.275275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:16.276387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:16.276387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:16.276387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:16.276387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:16.283218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 445/896] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:16.284756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80433f6-45e7-43e8-9891-0ef04840c84b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:16.324325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:16.333095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd542d5-3cc7-40fd-8bf4-ac2cd8777cd2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:16.367334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcf1dc-6c1a-4f11-8ea6-034fbbd3cea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:16.368052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcf1dc-6c1a-4f11-8ea6-034fbbd3cea5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.368339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcf1dc-6c1a-4f11-8ea6-034fbbd3cea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.368622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcf1dc-6c1a-4f11-8ea6-034fbbd3cea5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.369411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcf1dc-6c1a-4f11-8ea6-034fbbd3cea5 None admin] RBAC: Adding query filter params (interface=public, service_id=63d1d4df017444568adbbe12d3ef74dd, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:16.369787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcf1dc-6c1a-4f11-8ea6-034fbbd3cea5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:16.371281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcf1dc-6c1a-4f11-8ea6-034fbbd3cea5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:16.383973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 452/897] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 13 20:29:16 2022] GET /identity/v3/endpoints?service_id=63d1d4df017444568adbbe12d3ef74dd&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:16.384952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcf1dc-6c1a-4f11-8ea6-034fbbd3cea5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:16.396698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80433f6-45e7-43e8-9891-0ef04840c84b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:16.433574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfbb45-9eab-46a6-90ba-050c59d16881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:16.434507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfbb45-9eab-46a6-90ba-050c59d16881 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.435015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfbb45-9eab-46a6-90ba-050c59d16881 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.435692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfbb45-9eab-46a6-90ba-050c59d16881 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.436635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfbb45-9eab-46a6-90ba-050c59d16881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:16.437140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfbb45-9eab-46a6-90ba-050c59d16881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:16.438817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfbb45-9eab-46a6-90ba-050c59d16881 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:16.456939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 446/898] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:16.458628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfbb45-9eab-46a6-90ba-050c59d16881 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:16.483928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:16.495153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcf1dc-6c1a-4f11-8ea6-034fbbd3cea5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:16.529598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1c42-5ab4-4d93-afd7-8462042220f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:16.530277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1c42-5ab4-4d93-afd7-8462042220f3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.530563 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1c42-5ab4-4d93-afd7-8462042220f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.530839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1c42-5ab4-4d93-afd7-8462042220f3 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.531551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee1c42-5ab4-4d93-afd7-8462042220f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:16.532937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee1c42-5ab4-4d93-afd7-8462042220f3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:16.547808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 453/899] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 13 20:29:16 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:29:16.571157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfbb45-9eab-46a6-90ba-050c59d16881 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:16.610527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e13b7a-73a9-4c67-bc25-f13368189d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:16.611770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e13b7a-73a9-4c67-bc25-f13368189d3a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.612274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e13b7a-73a9-4c67-bc25-f13368189d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.612763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e13b7a-73a9-4c67-bc25-f13368189d3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.621181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e13b7a-73a9-4c67-bc25-f13368189d3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:16.623116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e13b7a-73a9-4c67-bc25-f13368189d3a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f6e13b7a-73a9-4c67-bc25-f13368189d3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:16.639632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:16.641000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:16.641873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:16.641873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:16.641873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:16.641873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:16.641873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:16.641873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:16.641873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:16.641873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:16.649403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 447/900] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:16.651340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1c42-5ab4-4d93-afd7-8462042220f3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:16.676090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:16.760092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1c42-5ab4-4d93-afd7-8462042220f3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:16.794801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f0337-e875-4900-a655-e2f0b0eb152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:16.795883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f0337-e875-4900-a655-e2f0b0eb152c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.796368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f0337-e875-4900-a655-e2f0b0eb152c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.796842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f0337-e875-4900-a655-e2f0b0eb152c None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.797832 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f0337-e875-4900-a655-e2f0b0eb152c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:16.798340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f0337-e875-4900-a655-e2f0b0eb152c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:16.800015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f0337-e875-4900-a655-e2f0b0eb152c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:16.819576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 454/901] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:16.822584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e13b7a-73a9-4c67-bc25-f13368189d3a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:16.907831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:16.927247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e13b7a-73a9-4c67-bc25-f13368189d3a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:16.939048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448cf49-a92f-4a03-acfb-62b78749a984 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.939335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448cf49-a92f-4a03-acfb-62b78749a984 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.939676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448cf49-a92f-4a03-acfb-62b78749a984 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.941060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 455/902] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:16.946312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a81ff9-619e-437e-b6f9-54757d4baaa4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.946605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a81ff9-619e-437e-b6f9-54757d4baaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.946885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a81ff9-619e-437e-b6f9-54757d4baaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.965030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-03a81ff9-619e-437e-b6f9-54757d4baaa4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:16.966434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-03a81ff9-619e-437e-b6f9-54757d4baaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:16.967696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3757623-ecc1-45b8-b869-da52e5eeab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:16.968610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3757623-ecc1-45b8-b869-da52e5eeab4c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:16.969098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3757623-ecc1-45b8-b869-da52e5eeab4c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:16.969563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3757623-ecc1-45b8-b869-da52e5eeab4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:16.970582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3757623-ecc1-45b8-b869-da52e5eeab4c None admin] RBAC: Adding query filter params (role.id=3c7426b3b0754e6089b5ccc02c8b3eec, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=95dcfd3d412a47298d7d5cf7b3b56f97) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:16.971089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3757623-ecc1-45b8-b869-da52e5eeab4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:16.972817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3757623-ecc1-45b8-b869-da52e5eeab4c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:16.978819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 448/903] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:16 2022] GET /identity/v3/role_assignments?user.id=95dcfd3d412a47298d7d5cf7b3b56f97&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=3c7426b3b0754e6089b5ccc02c8b3eec => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:17.041207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 456/904] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:17.066152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3757623-ecc1-45b8-b869-da52e5eeab4c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:17.174439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3757623-ecc1-45b8-b869-da52e5eeab4c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:17.214390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef10d50-8ad7-46c7-80d1-d6f3e37a3041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:17.215202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10d50-8ad7-46c7-80d1-d6f3e37a3041 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:17.215500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10d50-8ad7-46c7-80d1-d6f3e37a3041 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:17.215785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10d50-8ad7-46c7-80d1-d6f3e37a3041 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:17.216427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef10d50-8ad7-46c7-80d1-d6f3e37a3041 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:17.217809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef10d50-8ad7-46c7-80d1-d6f3e37a3041 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-eef10d50-8ad7-46c7-80d1-d6f3e37a3041 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:17.230388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:17.231332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:17.232383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 449/905] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Jul 13 20:29:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:17.238114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a81ff9-619e-437e-b6f9-54757d4baaa4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:17.250637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:17.349425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a81ff9-619e-437e-b6f9-54757d4baaa4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:17.385504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279abdc-59fb-4eca-8783-42eebcd3c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:17.386616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279abdc-59fb-4eca-8783-42eebcd3c3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:17.387168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279abdc-59fb-4eca-8783-42eebcd3c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:17.387674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279abdc-59fb-4eca-8783-42eebcd3c3e7 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:17.388578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279abdc-59fb-4eca-8783-42eebcd3c3e7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:17.389083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279abdc-59fb-4eca-8783-42eebcd3c3e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:17.390671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279abdc-59fb-4eca-8783-42eebcd3c3e7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:17.396210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 457/906] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:17.403528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef10d50-8ad7-46c7-80d1-d6f3e37a3041 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:17.413539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:17.510221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef10d50-8ad7-46c7-80d1-d6f3e37a3041 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:17.545660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99afaeb-9af9-4ef6-9154-760fb639c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:17.546704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99afaeb-9af9-4ef6-9154-760fb639c81a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:17.547198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99afaeb-9af9-4ef6-9154-760fb639c81a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:17.547711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99afaeb-9af9-4ef6-9154-760fb639c81a None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:17.548626 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99afaeb-9af9-4ef6-9154-760fb639c81a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:17.549128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99afaeb-9af9-4ef6-9154-760fb639c81a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:17.550719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99afaeb-9af9-4ef6-9154-760fb639c81a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:17.556504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 450/907] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:17.565161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279abdc-59fb-4eca-8783-42eebcd3c3e7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:17.580447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7092e5c-e402-4b2e-9050-05483c09e9ec None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:17.580762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7092e5c-e402-4b2e-9050-05483c09e9ec None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:17.581021 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7092e5c-e402-4b2e-9050-05483c09e9ec None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:17.582641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 451/908] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:17.587430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631ec64-6283-4640-8e99-4af932ec5b84 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:17.587759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631ec64-6283-4640-8e99-4af932ec5b84 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:17.588046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631ec64-6283-4640-8e99-4af932ec5b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:17.604975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5631ec64-6283-4640-8e99-4af932ec5b84 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:17.606174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5631ec64-6283-4640-8e99-4af932ec5b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:17.655526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:17.687872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279abdc-59fb-4eca-8783-42eebcd3c3e7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:17.699634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 452/909] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:17.719623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631ec64-6283-4640-8e99-4af932ec5b84 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:17.722207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:17.729099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae929b91-6bad-4ab3-901c-e32eb74329d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:17.729769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae929b91-6bad-4ab3-901c-e32eb74329d8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:17.730120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae929b91-6bad-4ab3-901c-e32eb74329d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:17.730328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae929b91-6bad-4ab3-901c-e32eb74329d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:17.731127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae929b91-6bad-4ab3-901c-e32eb74329d8 None admin] RBAC: Adding query filter params (interface=public, service_id=7c8ac7b986f143059bccbe2ad47089c5, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:17.731443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae929b91-6bad-4ab3-901c-e32eb74329d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:17.733331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae929b91-6bad-4ab3-901c-e32eb74329d8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:17.739785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 458/910] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 13 20:29:17 2022] GET /identity/v3/endpoints?service_id=7c8ac7b986f143059bccbe2ad47089c5&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:17.748404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae929b91-6bad-4ab3-901c-e32eb74329d8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:17.835990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631ec64-6283-4640-8e99-4af932ec5b84 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:17.871852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb4f77-1a65-4abf-bfce-c77732632ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:17.872561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb4f77-1a65-4abf-bfce-c77732632ebe None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:17.872850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb4f77-1a65-4abf-bfce-c77732632ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:17.873130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb4f77-1a65-4abf-bfce-c77732632ebe None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:17.873859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb4f77-1a65-4abf-bfce-c77732632ebe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:17.875243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb4f77-1a65-4abf-bfce-c77732632ebe None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:17.886911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae929b91-6bad-4ab3-901c-e32eb74329d8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:17.891713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 453/911] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:29:17 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:17.937700 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74130-f8f8-4c70-8e30-dbfaced7f621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:17.938686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74130-f8f8-4c70-8e30-dbfaced7f621 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:17.939166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74130-f8f8-4c70-8e30-dbfaced7f621 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:17.939622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74130-f8f8-4c70-8e30-dbfaced7f621 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:17.940483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a74130-f8f8-4c70-8e30-dbfaced7f621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:17.942108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a74130-f8f8-4c70-8e30-dbfaced7f621 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:17.943710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:17.950476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 459/912] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 13 20:29:17 2022] GET /identity/v3/services => generated 1497 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:29:18.681610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222799e8-a464-4f55-a57a-082915133d90 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:18.682056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222799e8-a464-4f55-a57a-082915133d90 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:18.682284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222799e8-a464-4f55-a57a-082915133d90 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:18.683926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 454/913] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:18.689222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e25e55-4927-4773-971b-3e588c48566d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:18.689659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e25e55-4927-4773-971b-3e588c48566d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:18.689846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e25e55-4927-4773-971b-3e588c48566d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:18.706674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b1e25e55-4927-4773-971b-3e588c48566d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:18.707873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b1e25e55-4927-4773-971b-3e588c48566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:18.778839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 460/914] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:18.800853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222799e8-a464-4f55-a57a-082915133d90 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:18.861463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:18.902333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222799e8-a464-4f55-a57a-082915133d90 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:18.936055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4323a-b105-411a-9769-e32868c46065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:18.936822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4323a-b105-411a-9769-e32868c46065 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:18.937123 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4323a-b105-411a-9769-e32868c46065 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:18.937393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4323a-b105-411a-9769-e32868c46065 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:18.938040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4323a-b105-411a-9769-e32868c46065 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:18.939418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4323a-b105-411a-9769-e32868c46065 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-04c4323a-b105-411a-9769-e32868c46065 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:18.951649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:18.952597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:18.953413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 455/915] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:29:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:18.959086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e25e55-4927-4773-971b-3e588c48566d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:19.065576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e25e55-4927-4773-971b-3e588c48566d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:19.092805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:19.107269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffc6f3-e31e-48e8-af64-157810e43dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:19.107994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffc6f3-e31e-48e8-af64-157810e43dcd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.108277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffc6f3-e31e-48e8-af64-157810e43dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.108553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffc6f3-e31e-48e8-af64-157810e43dcd None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:19.109274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffc6f3-e31e-48e8-af64-157810e43dcd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:19.109639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffc6f3-e31e-48e8-af64-157810e43dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:19.111083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffc6f3-e31e-48e8-af64-157810e43dcd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:19.117050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 461/916] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:29:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:19.122733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4323a-b105-411a-9769-e32868c46065 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:19.162248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643c689-3655-446c-b2bb-d0deaa214535 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.162582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643c689-3655-446c-b2bb-d0deaa214535 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.162857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643c689-3655-446c-b2bb-d0deaa214535 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:19.166075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 462/917] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:19.172974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6303d-e790-4a85-9d53-a1dcb32e94db None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.173263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6303d-e790-4a85-9d53-a1dcb32e94db None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.173542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6303d-e790-4a85-9d53-a1dcb32e94db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:19.193674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bac6303d-e790-4a85-9d53-a1dcb32e94db None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:19.195413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bac6303d-e790-4a85-9d53-a1dcb32e94db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:19.236271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4323a-b105-411a-9769-e32868c46065 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:19.261556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:19.268014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 463/918] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:19.273899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2395b-249a-43bb-b7c1-4f66656eeef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:19.274791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2395b-249a-43bb-b7c1-4f66656eeef6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.275285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2395b-249a-43bb-b7c1-4f66656eeef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.275859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2395b-249a-43bb-b7c1-4f66656eeef6 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:19.276753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2395b-249a-43bb-b7c1-4f66656eeef6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:19.277249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2395b-249a-43bb-b7c1-4f66656eeef6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:19.278819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2395b-249a-43bb-b7c1-4f66656eeef6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:19.284328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 456/919] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 13 20:29:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:19.290126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6303d-e790-4a85-9d53-a1dcb32e94db None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:19.291734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2395b-249a-43bb-b7c1-4f66656eeef6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:19.404308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6303d-e790-4a85-9d53-a1dcb32e94db None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:19.410300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2395b-249a-43bb-b7c1-4f66656eeef6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:19.427412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:19.443117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff3d6b-18be-4c43-b042-2a5579172903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:19.444185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff3d6b-18be-4c43-b042-2a5579172903 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.444648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff3d6b-18be-4c43-b042-2a5579172903 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.445143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff3d6b-18be-4c43-b042-2a5579172903 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:19.446086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff3d6b-18be-4c43-b042-2a5579172903 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:19.448026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff3d6b-18be-4c43-b042-2a5579172903 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:19.452801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4482b94-c15d-4385-98b7-275979197700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:19.453959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4482b94-c15d-4385-98b7-275979197700 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.454309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4482b94-c15d-4385-98b7-275979197700 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.454658 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4482b94-c15d-4385-98b7-275979197700 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:19.470496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4482b94-c15d-4385-98b7-275979197700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:19.472720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 464/920] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 13 20:29:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b4482b94-c15d-4385-98b7-275979197700 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:19.486941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:19.488050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:19.488877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:19.496978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 457/921] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 13 20:29:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:19.498895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff3d6b-18be-4c43-b042-2a5579172903 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:19.626545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff3d6b-18be-4c43-b042-2a5579172903 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:19.662251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d28539-4e57-447f-abee-f890ef4eb3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:19.662937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d28539-4e57-447f-abee-f890ef4eb3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.663225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d28539-4e57-447f-abee-f890ef4eb3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.663527 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d28539-4e57-447f-abee-f890ef4eb3e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:19.664285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d28539-4e57-447f-abee-f890ef4eb3e6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:19.664596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d28539-4e57-447f-abee-f890ef4eb3e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:19.665982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d28539-4e57-447f-abee-f890ef4eb3e6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:19.674897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 465/922] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 13 20:29:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:19.680467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4482b94-c15d-4385-98b7-275979197700 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:19.739963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:19.787846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4482b94-c15d-4385-98b7-275979197700 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:19.822174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf7ba1-0003-472c-8dec-d6261474192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:19.824169 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf7ba1-0003-472c-8dec-d6261474192c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.824630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf7ba1-0003-472c-8dec-d6261474192c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.825093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf7ba1-0003-472c-8dec-d6261474192c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:19.837240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf7ba1-0003-472c-8dec-d6261474192c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:19.839018 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf7ba1-0003-472c-8dec-d6261474192c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bdcf7ba1-0003-472c-8dec-d6261474192c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:19.856920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:19.857836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:19.858715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:19.858715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:19.858715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:19.858715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:19.863567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 458/923] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:19.866577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d28539-4e57-447f-abee-f890ef4eb3e6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:19.904026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:19.975047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d28539-4e57-447f-abee-f890ef4eb3e6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:19.978129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3b729-ce82-4f42-8779-b3533c261ebd None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.978421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3b729-ce82-4f42-8779-b3533c261ebd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.978699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3b729-ce82-4f42-8779-b3533c261ebd None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:19.980436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 459/924] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:19.985872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8fbae-6fbe-4a91-934d-7fa23bb47140 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:19.986195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8fbae-6fbe-4a91-934d-7fa23bb47140 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:19.986566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8fbae-6fbe-4a91-934d-7fa23bb47140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.004118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-58a8fbae-6fbe-4a91-934d-7fa23bb47140 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:20.005328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-58a8fbae-6fbe-4a91-934d-7fa23bb47140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:20.010265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e11b8-4af4-4cab-91f8-214b6b513986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.010937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e11b8-4af4-4cab-91f8-214b6b513986 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.011217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e11b8-4af4-4cab-91f8-214b6b513986 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.011529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e11b8-4af4-4cab-91f8-214b6b513986 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.012240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e11b8-4af4-4cab-91f8-214b6b513986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:20.012544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e11b8-4af4-4cab-91f8-214b6b513986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.013905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e11b8-4af4-4cab-91f8-214b6b513986 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.030205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 466/925] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:20.036192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e11b8-4af4-4cab-91f8-214b6b513986 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.090331 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 460/926] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:20.092171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a227a-f05c-45c6-890f-8507d0cff6d1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.092663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a227a-f05c-45c6-890f-8507d0cff6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.093134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a227a-f05c-45c6-890f-8507d0cff6d1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.094845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 461/927] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:20.100587 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5834aa4-f0da-4c66-a565-8db522e4515a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.100872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5834aa4-f0da-4c66-a565-8db522e4515a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.101148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5834aa4-f0da-4c66-a565-8db522e4515a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.122248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d5834aa4-f0da-4c66-a565-8db522e4515a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:20.123423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d5834aa4-f0da-4c66-a565-8db522e4515a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:20.149800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:20.151200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:20.157294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e11b8-4af4-4cab-91f8-214b6b513986 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:20.192522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 462/928] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:20.193178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5834aa4-f0da-4c66-a565-8db522e4515a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.200541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74350020-b63d-455c-b05b-571f6f0d052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.202165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74350020-b63d-455c-b05b-571f6f0d052a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.202660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74350020-b63d-455c-b05b-571f6f0d052a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.203628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74350020-b63d-455c-b05b-571f6f0d052a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.213806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74350020-b63d-455c-b05b-571f6f0d052a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.215248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74350020-b63d-455c-b05b-571f6f0d052a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-74350020-b63d-455c-b05b-571f6f0d052a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:20.231343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:20.232358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 467/929] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:20.238956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74350020-b63d-455c-b05b-571f6f0d052a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.306098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5834aa4-f0da-4c66-a565-8db522e4515a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:20.341013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74350020-b63d-455c-b05b-571f6f0d052a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:20.342941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af723073-3738-4f29-8ffe-a51b614e24f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.343854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af723073-3738-4f29-8ffe-a51b614e24f5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.344151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af723073-3738-4f29-8ffe-a51b614e24f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.344435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af723073-3738-4f29-8ffe-a51b614e24f5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.345101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af723073-3738-4f29-8ffe-a51b614e24f5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.346494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af723073-3738-4f29-8ffe-a51b614e24f5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-af723073-3738-4f29-8ffe-a51b614e24f5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:20.358448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:20.359428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:20.360284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 463/930] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:20.361083 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af723073-3738-4f29-8ffe-a51b614e24f5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.383053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcf85d-fedb-44e5-b4d8-a416332fa660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.383909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcf85d-fedb-44e5-b4d8-a416332fa660 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.384198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcf85d-fedb-44e5-b4d8-a416332fa660 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.384475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcf85d-fedb-44e5-b4d8-a416332fa660 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.385178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcf85d-fedb-44e5-b4d8-a416332fa660 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.386550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcf85d-fedb-44e5-b4d8-a416332fa660 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.391490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-81bcf85d-fedb-44e5-b4d8-a416332fa660 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:20.398801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:20.399972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:29:20.401077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:20.404929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 468/931] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:20.406064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcf85d-fedb-44e5-b4d8-a416332fa660 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.428190 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:20.473697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af723073-3738-4f29-8ffe-a51b614e24f5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:20.511645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaaae3-8fa7-4bcd-9ad3-6f658da150a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.512326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaaae3-8fa7-4bcd-9ad3-6f658da150a3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.512616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaaae3-8fa7-4bcd-9ad3-6f658da150a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.512895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaaae3-8fa7-4bcd-9ad3-6f658da150a3 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.513745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaaae3-8fa7-4bcd-9ad3-6f658da150a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:20.514058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaaae3-8fa7-4bcd-9ad3-6f658da150a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.514240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcf85d-fedb-44e5-b4d8-a416332fa660 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:20.515527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaaae3-8fa7-4bcd-9ad3-6f658da150a3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.533591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 464/932] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:20.538769 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaaae3-8fa7-4bcd-9ad3-6f658da150a3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.549590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4377c65-320c-4514-956e-b1a7d275c9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.550263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4377c65-320c-4514-956e-b1a7d275c9cc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.550551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4377c65-320c-4514-956e-b1a7d275c9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.550832 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4377c65-320c-4514-956e-b1a7d275c9cc None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.551571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4377c65-320c-4514-956e-b1a7d275c9cc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:20.551884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4377c65-320c-4514-956e-b1a7d275c9cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.553251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4377c65-320c-4514-956e-b1a7d275c9cc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.562532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 469/933] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:20.562532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4377c65-320c-4514-956e-b1a7d275c9cc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.632930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:20.648513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:20.652785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaaae3-8fa7-4bcd-9ad3-6f658da150a3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:20.667525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4377c65-320c-4514-956e-b1a7d275c9cc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:20.688906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81c14d-49c7-427f-a200-31c5a97125ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.689577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81c14d-49c7-427f-a200-31c5a97125ff None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.689865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81c14d-49c7-427f-a200-31c5a97125ff None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.690143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81c14d-49c7-427f-a200-31c5a97125ff None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.690853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81c14d-49c7-427f-a200-31c5a97125ff None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:20.691162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81c14d-49c7-427f-a200-31c5a97125ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.692593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81c14d-49c7-427f-a200-31c5a97125ff None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.702095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 465/934] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:20.702095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81c14d-49c7-427f-a200-31c5a97125ff None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.709504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad319aa6-49a3-48ed-9f44-44d7fe4a2fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.710635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad319aa6-49a3-48ed-9f44-44d7fe4a2fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.711072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad319aa6-49a3-48ed-9f44-44d7fe4a2fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.711360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad319aa6-49a3-48ed-9f44-44d7fe4a2fb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.722574 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad319aa6-49a3-48ed-9f44-44d7fe4a2fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.724271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad319aa6-49a3-48ed-9f44-44d7fe4a2fb2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ad319aa6-49a3-48ed-9f44-44d7fe4a2fb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:20.745773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:20.746864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:20.747862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:20.747862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:20.747862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:20.747862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:20.751692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 470/935] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:20.752517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad319aa6-49a3-48ed-9f44-44d7fe4a2fb2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.806810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81c14d-49c7-427f-a200-31c5a97125ff None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:20.844239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ade55-fa2f-4f7a-8057-78ff773ed18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.846025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ade55-fa2f-4f7a-8057-78ff773ed18d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.846526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ade55-fa2f-4f7a-8057-78ff773ed18d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.846878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ade55-fa2f-4f7a-8057-78ff773ed18d None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.847764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ade55-fa2f-4f7a-8057-78ff773ed18d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:20.848149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ade55-fa2f-4f7a-8057-78ff773ed18d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.849460 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ade55-fa2f-4f7a-8057-78ff773ed18d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.854610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 466/936] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:20.858498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad319aa6-49a3-48ed-9f44-44d7fe4a2fb2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:20.860284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ade55-fa2f-4f7a-8057-78ff773ed18d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.875979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:20.895982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea80135-c39d-4ef7-8e17-296ae45ed9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:20.896676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea80135-c39d-4ef7-8e17-296ae45ed9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:20.896964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea80135-c39d-4ef7-8e17-296ae45ed9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:20.897236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea80135-c39d-4ef7-8e17-296ae45ed9c5 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:20.898083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea80135-c39d-4ef7-8e17-296ae45ed9c5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:20.898270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea80135-c39d-4ef7-8e17-296ae45ed9c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:20.899640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea80135-c39d-4ef7-8e17-296ae45ed9c5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:20.902035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:20.905640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 471/937] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:20.906449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea80135-c39d-4ef7-8e17-296ae45ed9c5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:20.968783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ade55-fa2f-4f7a-8057-78ff773ed18d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:21.006668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10a50e-8a99-4d69-991b-9dd34256e7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:21.008293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10a50e-8a99-4d69-991b-9dd34256e7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.008751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10a50e-8a99-4d69-991b-9dd34256e7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.009171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10a50e-8a99-4d69-991b-9dd34256e7e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.010064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10a50e-8a99-4d69-991b-9dd34256e7e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:21.010541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10a50e-8a99-4d69-991b-9dd34256e7e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:21.010961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea80135-c39d-4ef7-8e17-296ae45ed9c5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:21.012386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10a50e-8a99-4d69-991b-9dd34256e7e3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:21.029139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 467/938] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:21.029139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10a50e-8a99-4d69-991b-9dd34256e7e3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:21.045976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d34b45-fe1e-46dc-abfa-fc19d4a3c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:21.046641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d34b45-fe1e-46dc-abfa-fc19d4a3c0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.046930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d34b45-fe1e-46dc-abfa-fc19d4a3c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.047208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d34b45-fe1e-46dc-abfa-fc19d4a3c0b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.050881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d34b45-fe1e-46dc-abfa-fc19d4a3c0b9 None admin] RBAC: Adding query filter params (interface=public, service_id=d04275f94c404cce91ee16654a02c891, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:21.050881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d34b45-fe1e-46dc-abfa-fc19d4a3c0b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:21.050881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d34b45-fe1e-46dc-abfa-fc19d4a3c0b9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:21.055492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 472/939] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 13 20:29:20 2022] GET /identity/v3/endpoints?service_id=d04275f94c404cce91ee16654a02c891&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:21.056374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d34b45-fe1e-46dc-abfa-fc19d4a3c0b9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:21.114837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:21.120945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:21.147693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10a50e-8a99-4d69-991b-9dd34256e7e3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:21.162976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d34b45-fe1e-46dc-abfa-fc19d4a3c0b9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:21.182835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3d982-ed68-4b96-b4c4-abf2503efa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:21.183618 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3d982-ed68-4b96-b4c4-abf2503efa10 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.183975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3d982-ed68-4b96-b4c4-abf2503efa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.184349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3d982-ed68-4b96-b4c4-abf2503efa10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.185171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3d982-ed68-4b96-b4c4-abf2503efa10 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:21.186638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3d982-ed68-4b96-b4c4-abf2503efa10 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:21.198454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d43992-60e8-410c-8120-dcfc93782945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:21.199353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d43992-60e8-410c-8120-dcfc93782945 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.199746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d43992-60e8-410c-8120-dcfc93782945 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.200073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d43992-60e8-410c-8120-dcfc93782945 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.207600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d43992-60e8-410c-8120-dcfc93782945 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:21.209395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d43992-60e8-410c-8120-dcfc93782945 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:21.223505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 468/940] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 13 20:29:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:21.223505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3d982-ed68-4b96-b4c4-abf2503efa10 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-20d43992-60e8-410c-8120-dcfc93782945 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:21.224424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:21.225290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:21.226251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:21.226251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:21.226251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:21.226251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:21.226251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:21.226251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:21.226251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:21.226251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:21.226251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 473/941] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:21.233469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d43992-60e8-410c-8120-dcfc93782945 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:21.340326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3d982-ed68-4b96-b4c4-abf2503efa10 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:21.361827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d43992-60e8-410c-8120-dcfc93782945 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:21.367540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:21.382118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068002c-87ec-4f69-9c78-eb246c67bf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:21.382799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068002c-87ec-4f69-9c78-eb246c67bf17 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.383088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068002c-87ec-4f69-9c78-eb246c67bf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.383363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068002c-87ec-4f69-9c78-eb246c67bf17 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.384180 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068002c-87ec-4f69-9c78-eb246c67bf17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:21.385673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068002c-87ec-4f69-9c78-eb246c67bf17 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:21.394005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 469/942] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 13 20:29:21 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:29:21.398936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:21.412017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87971b2-66e3-4d42-82f2-f6d9305ea37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:21.413532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87971b2-66e3-4d42-82f2-f6d9305ea37e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.413532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87971b2-66e3-4d42-82f2-f6d9305ea37e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.413532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87971b2-66e3-4d42-82f2-f6d9305ea37e None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.414226 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87971b2-66e3-4d42-82f2-f6d9305ea37e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:21.414603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87971b2-66e3-4d42-82f2-f6d9305ea37e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:21.416457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87971b2-66e3-4d42-82f2-f6d9305ea37e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:21.432827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 474/943] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:21.439998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068002c-87ec-4f69-9c78-eb246c67bf17 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:21.562273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068002c-87ec-4f69-9c78-eb246c67bf17 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:21.579005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdeec8-1fd9-4bae-98c7-3e9e9bcd4708 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.579686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdeec8-1fd9-4bae-98c7-3e9e9bcd4708 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.580159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdeec8-1fd9-4bae-98c7-3e9e9bcd4708 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.581818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 475/944] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:21.589055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf16cb5-0933-4474-829b-c35df1586ea3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.589807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf16cb5-0933-4474-829b-c35df1586ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.590308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf16cb5-0933-4474-829b-c35df1586ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.599256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3fbf4-447c-4354-a8b3-849fa315a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:21.601383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3fbf4-447c-4354-a8b3-849fa315a0c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.601818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3fbf4-447c-4354-a8b3-849fa315a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.602255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3fbf4-447c-4354-a8b3-849fa315a0c5 None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/95dcfd3d412a47298d7d5cf7b3b56f97/roles/3c7426b3b0754e6089b5ccc02c8b3eec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.604679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3fbf4-447c-4354-a8b3-849fa315a0c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=95dcfd3d412a47298d7d5cf7b3b56f97, role_id=3c7426b3b0754e6089b5ccc02c8b3eec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:21.606313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3fbf4-447c-4354-a8b3-849fa315a0c5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:21.620623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8bf16cb5-0933-4474-829b-c35df1586ea3 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:21.622446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8bf16cb5-0933-4474-829b-c35df1586ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:21.627293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 470/945] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:29:21 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/95dcfd3d412a47298d7d5cf7b3b56f97/roles/3c7426b3b0754e6089b5ccc02c8b3eec => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:29:21.709895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 476/946] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:21.734565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3fbf4-447c-4354-a8b3-849fa315a0c5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:21.792273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:21.846651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3fbf4-447c-4354-a8b3-849fa315a0c5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:21.882568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:21.883160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:21.883466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:21.883757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:21.884508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:21.886016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:21.887394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WARNING keystone.server.flask.common [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 13 20:29:21.906362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.sql.core [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 20:29:21.906362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 20:29:21.906362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [parameters: {'id': '48d9e142bf8d4aa8abe76335ef7e8618', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 20:29:21.906362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48d9e142bf8d4aa8abe76335ef7e8618.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48d9e142bf8d4aa8abe76335ef7e8618. Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:29:21.914386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 13 20:29:21.915172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application yield resource Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise self.value Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application session.commit() Jul 13 20:29:21.916738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.flush() Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application compat.raise_( Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 20:29:21.918479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 20:29:21.920578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:29:21.925260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [parameters: {'id': '48d9e142bf8d4aa8abe76335ef7e8618', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:21.927401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:21.929330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48d9e142bf8d4aa8abe76335ef7e8618. Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:21.931408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 471/947] 10.222.0.47 () {60 vars in 1140 bytes} [Wed Jul 13 20:29:21 2022] POST /identity/v3/projects => generated 262 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:21.932753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf16cb5-0933-4474-829b-c35df1586ea3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:22.034603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf16cb5-0933-4474-829b-c35df1586ea3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:22.050129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:22.071742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e0a4-f569-48d7-be2e-f28aa0f20d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:22.072948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e0a4-f569-48d7-be2e-f28aa0f20d6a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:22.073239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e0a4-f569-48d7-be2e-f28aa0f20d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:22.073521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e0a4-f569-48d7-be2e-f28aa0f20d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:22.080839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47e0a4-f569-48d7-be2e-f28aa0f20d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:22.082354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47e0a4-f569-48d7-be2e-f28aa0f20d6a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fb47e0a4-f569-48d7-be2e-f28aa0f20d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:22.096381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:22.098422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:22.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:22.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:22.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:22.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:22.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:22.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:22.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:22.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:22.100565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 477/948] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:22.104051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:22.216394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ae746-56fb-4979-9663-3b3ea5d15596 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:22.218855 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:22.251114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8c7da-328b-4d92-b297-0580c1b3cd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:22.251797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8c7da-328b-4d92-b297-0580c1b3cd50 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:22.252087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8c7da-328b-4d92-b297-0580c1b3cd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:22.252369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8c7da-328b-4d92-b297-0580c1b3cd50 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:22.253128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8c7da-328b-4d92-b297-0580c1b3cd50 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:22.253441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8c7da-328b-4d92-b297-0580c1b3cd50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:22.255097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8c7da-328b-4d92-b297-0580c1b3cd50 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:22.275276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 472/949] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:23.417240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abef900-8ecd-4ecb-989c-0d882ad5df31 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.417240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abef900-8ecd-4ecb-989c-0d882ad5df31 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.417240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abef900-8ecd-4ecb-989c-0d882ad5df31 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.417240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 478/950] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:23.426224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ebf29-ba75-4ed7-b5ae-bc5284cf8cc3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.426224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ebf29-ba75-4ed7-b5ae-bc5284cf8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.426224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ebf29-ba75-4ed7-b5ae-bc5284cf8cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.443071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f66ebf29-ba75-4ed7-b5ae-bc5284cf8cc3 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:23.444918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f66ebf29-ba75-4ed7-b5ae-bc5284cf8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:23.454898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ec690-56e0-4913-baa8-2fd354f97740 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.455207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ec690-56e0-4913-baa8-2fd354f97740 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.455489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ec690-56e0-4913-baa8-2fd354f97740 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.457249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 479/951] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:23.462262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21361-7770-4562-b470-9721b458cf5f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.462556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21361-7770-4562-b470-9721b458cf5f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.462924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21361-7770-4562-b470-9721b458cf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.479969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4ba21361-7770-4562-b470-9721b458cf5f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:23.481186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4ba21361-7770-4562-b470-9721b458cf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:23.527564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 473/952] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:23.542249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ebf29-ba75-4ed7-b5ae-bc5284cf8cc3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:23.551117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 480/953] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:23.574923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21361-7770-4562-b470-9721b458cf5f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:23.636621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:23.643391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ebf29-ba75-4ed7-b5ae-bc5284cf8cc3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:23.679677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21361-7770-4562-b470-9721b458cf5f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:23.682913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:23.686321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b803cb9-4c31-43b9-b726-18ac268aa831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:23.687140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b803cb9-4c31-43b9-b726-18ac268aa831 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.687425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b803cb9-4c31-43b9-b726-18ac268aa831 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.687767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b803cb9-4c31-43b9-b726-18ac268aa831 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.688454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b803cb9-4c31-43b9-b726-18ac268aa831 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:23.691072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b803cb9-4c31-43b9-b726-18ac268aa831 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9b803cb9-4c31-43b9-b726-18ac268aa831 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:23.709308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:23.710349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:29:23.711396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:23.712245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 474/954] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:23.717427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b803cb9-4c31-43b9-b726-18ac268aa831 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:23.723225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690b826-4953-4f73-85cf-b1d91d482208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:23.724075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690b826-4953-4f73-85cf-b1d91d482208 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.724361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690b826-4953-4f73-85cf-b1d91d482208 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.724636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690b826-4953-4f73-85cf-b1d91d482208 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.725294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690b826-4953-4f73-85cf-b1d91d482208 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:23.726668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690b826-4953-4f73-85cf-b1d91d482208 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7690b826-4953-4f73-85cf-b1d91d482208 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:23.739875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:23.741112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:23.742032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 481/955] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:23.743164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30172d-20d3-4f26-935e-db43683ddae6 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.743164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30172d-20d3-4f26-935e-db43683ddae6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.743164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30172d-20d3-4f26-935e-db43683ddae6 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.744497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 482/956] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:23.747130 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30172d-20d3-4f26-935e-db43683ddae6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:23.828917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b803cb9-4c31-43b9-b726-18ac268aa831 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:23.849224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30172d-20d3-4f26-935e-db43683ddae6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:23.865232 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9ec1e-451e-4e7b-8abe-a2e896e1c642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:23.865929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9ec1e-451e-4e7b-8abe-a2e896e1c642 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.866187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9ec1e-451e-4e7b-8abe-a2e896e1c642 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.866466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9ec1e-451e-4e7b-8abe-a2e896e1c642 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.867169 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9ec1e-451e-4e7b-8abe-a2e896e1c642 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:23.867496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9ec1e-451e-4e7b-8abe-a2e896e1c642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:23.868886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9ec1e-451e-4e7b-8abe-a2e896e1c642 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:23.873767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 475/957] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:23.878724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9800551-180f-4a65-b2ac-9aa5be0e2f21 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.878724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9800551-180f-4a65-b2ac-9aa5be0e2f21 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.878724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9800551-180f-4a65-b2ac-9aa5be0e2f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.889119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca8cc5-e062-4a42-9404-c1120a364392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:23.889866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca8cc5-e062-4a42-9404-c1120a364392 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:23.890219 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca8cc5-e062-4a42-9404-c1120a364392 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:23.890564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca8cc5-e062-4a42-9404-c1120a364392 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:23.891341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca8cc5-e062-4a42-9404-c1120a364392 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:23.891744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca8cc5-e062-4a42-9404-c1120a364392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:23.893180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca8cc5-e062-4a42-9404-c1120a364392 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:23.894679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d9800551-180f-4a65-b2ac-9aa5be0e2f21 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:23.895903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d9800551-180f-4a65-b2ac-9aa5be0e2f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:23.898416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 483/958] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:23.899066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca8cc5-e062-4a42-9404-c1120a364392 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:23.919184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:23.933368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:23.972948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 476/959] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:23.982892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9800551-180f-4a65-b2ac-9aa5be0e2f21 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:24.009715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca8cc5-e062-4a42-9404-c1120a364392 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:24.047635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02613f22-7a2f-46d7-910a-718d597f82a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:24.048492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02613f22-7a2f-46d7-910a-718d597f82a8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.048836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02613f22-7a2f-46d7-910a-718d597f82a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.049174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02613f22-7a2f-46d7-910a-718d597f82a8 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.049993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02613f22-7a2f-46d7-910a-718d597f82a8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:24.050361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02613f22-7a2f-46d7-910a-718d597f82a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:24.051853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02613f22-7a2f-46d7-910a-718d597f82a8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:24.057811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 484/960] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:24.059168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02613f22-7a2f-46d7-910a-718d597f82a8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:24.137187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9800551-180f-4a65-b2ac-9aa5be0e2f21 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:24.171890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:24.179648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02613f22-7a2f-46d7-910a-718d597f82a8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:24.186496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc7c53-2121-4a6f-9862-9f359a87dc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:24.187424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc7c53-2121-4a6f-9862-9f359a87dc41 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.187902 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc7c53-2121-4a6f-9862-9f359a87dc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.188983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc7c53-2121-4a6f-9862-9f359a87dc41 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.189866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc7c53-2121-4a6f-9862-9f359a87dc41 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:24.190332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc7c53-2121-4a6f-9862-9f359a87dc41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:24.193168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc7c53-2121-4a6f-9862-9f359a87dc41 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:24.201408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 477/961] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:24.207115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc7c53-2121-4a6f-9862-9f359a87dc41 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:24.226356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017973e2-a3e6-41ee-86bf-4f6242f53c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:24.227585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017973e2-a3e6-41ee-86bf-4f6242f53c1a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.228057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017973e2-a3e6-41ee-86bf-4f6242f53c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.228517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017973e2-a3e6-41ee-86bf-4f6242f53c1a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.236036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:24.241708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017973e2-a3e6-41ee-86bf-4f6242f53c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-017973e2-a3e6-41ee-86bf-4f6242f53c1a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:24.251276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:24.252252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:24.253074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:24.260570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 485/962] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 13 20:29:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:24.261714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017973e2-a3e6-41ee-86bf-4f6242f53c1a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:24.334397 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc7c53-2121-4a6f-9862-9f359a87dc41 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:24.370657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb0cfa-ecda-4ca5-bfd8-4c96257d0d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:24.371724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb0cfa-ecda-4ca5-bfd8-4c96257d0d60 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.372098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb0cfa-ecda-4ca5-bfd8-4c96257d0d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.372443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb0cfa-ecda-4ca5-bfd8-4c96257d0d60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.373246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb0cfa-ecda-4ca5-bfd8-4c96257d0d60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:24.374687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb0cfa-ecda-4ca5-bfd8-4c96257d0d60 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:24.378760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017973e2-a3e6-41ee-86bf-4f6242f53c1a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:24.396011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 478/963] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 13 20:29:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:24.397349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffb5ba-ab6c-42d4-9d56-cc47104e017e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.397643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffb5ba-ab6c-42d4-9d56-cc47104e017e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.397928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffb5ba-ab6c-42d4-9d56-cc47104e017e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.399392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 479/964] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:24.405140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d1375-f673-4ff1-b106-2820eda68605 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.405439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d1375-f673-4ff1-b106-2820eda68605 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.405728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d1375-f673-4ff1-b106-2820eda68605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.423490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c52d1375-f673-4ff1-b106-2820eda68605 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:24.424739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c52d1375-f673-4ff1-b106-2820eda68605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:24.434248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07331ad-ea14-47f1-bc8f-fa5ed956a8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:24.435032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07331ad-ea14-47f1-bc8f-fa5ed956a8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.435604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07331ad-ea14-47f1-bc8f-fa5ed956a8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.435957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07331ad-ea14-47f1-bc8f-fa5ed956a8d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.436856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07331ad-ea14-47f1-bc8f-fa5ed956a8d1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:24.437174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07331ad-ea14-47f1-bc8f-fa5ed956a8d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:24.438778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07331ad-ea14-47f1-bc8f-fa5ed956a8d1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:24.449152 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 486/965] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 13 20:29:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:24.455541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07331ad-ea14-47f1-bc8f-fa5ed956a8d1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:24.475490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:24.511504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 480/966] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:24.538817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d1375-f673-4ff1-b106-2820eda68605 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:24.544629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:24.606655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07331ad-ea14-47f1-bc8f-fa5ed956a8d1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:24.650322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217d781-7609-4a66-839d-cb40ca27626a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:24.651751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d781-7609-4a66-839d-cb40ca27626a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.652038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d781-7609-4a66-839d-cb40ca27626a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.652588 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d781-7609-4a66-839d-cb40ca27626a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.663037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d1375-f673-4ff1-b106-2820eda68605 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:24.667434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217d781-7609-4a66-839d-cb40ca27626a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:24.668931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217d781-7609-4a66-839d-cb40ca27626a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f217d781-7609-4a66-839d-cb40ca27626a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:24.690892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:24.691867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:24.692772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:24.692772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:24.692772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:24.692772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:24.693411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 487/967] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:24.699079 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217d781-7609-4a66-839d-cb40ca27626a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:24.703672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:24.704101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.704384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.704657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.705373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:24.706810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:24.722136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.sql.core [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 20:29:24.722136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 20:29:24.722136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [parameters: {'user_id': 'f2467482636a48bda887670aaa5c0861', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 20:29:24.722136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 13 20:29:24.732043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:29:24.732983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application yield resource Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise self.value Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application session.commit() Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 20:29:24.733915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.flush() Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application compat.raise_( Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 20:29:24.734800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:29:24.735738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:29:24.736624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f2467482636a48bda887670aaa5c0861', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:24.737540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 13 20:29:24.738357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 13 20:29:24.739305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 13 20:29:24.739305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 13 20:29:24.739305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 20:29:24.739305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 20:29:24.739305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 20:29:24.739305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:24.739305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 481/968] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 13 20:29:24 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:24.741479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:24.789326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:24.824926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:24.842291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217d781-7609-4a66-839d-cb40ca27626a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:24.874756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2da7e-2dd9-4c90-828a-ae37f53e8312 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:24.888511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea448a-87ff-4b6d-97d1-7e582d6b1d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:24.889688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea448a-87ff-4b6d-97d1-7e582d6b1d75 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.890195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea448a-87ff-4b6d-97d1-7e582d6b1d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.890671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea448a-87ff-4b6d-97d1-7e582d6b1d75 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.891818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea448a-87ff-4b6d-97d1-7e582d6b1d75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:24.892683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea448a-87ff-4b6d-97d1-7e582d6b1d75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:24.894558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea448a-87ff-4b6d-97d1-7e582d6b1d75 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:24.910142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 488/969] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:24.915592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea448a-87ff-4b6d-97d1-7e582d6b1d75 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:24.927445 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d0cbd-2e34-4aec-867b-774a9c521aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:24.928325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d0cbd-2e34-4aec-867b-774a9c521aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:24.929073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d0cbd-2e34-4aec-867b-774a9c521aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:24.929357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d0cbd-2e34-4aec-867b-774a9c521aa4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:24.940656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d0cbd-2e34-4aec-867b-774a9c521aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:24.942375 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d0cbd-2e34-4aec-867b-774a9c521aa4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f37d0cbd-2e34-4aec-867b-774a9c521aa4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:24.960605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:24.961613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:29:24.962563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:24.970012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 482/970] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:29:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:24.971127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d0cbd-2e34-4aec-867b-774a9c521aa4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:25.039073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea448a-87ff-4b6d-97d1-7e582d6b1d75 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:25.072074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:25.080202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d0cbd-2e34-4aec-867b-774a9c521aa4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:25.084623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc90f47-8a41-4517-bf12-6cfe7a10a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:25.085610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc90f47-8a41-4517-bf12-6cfe7a10a534 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:25.086033 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc90f47-8a41-4517-bf12-6cfe7a10a534 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:25.086463 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc90f47-8a41-4517-bf12-6cfe7a10a534 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:25.096352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc90f47-8a41-4517-bf12-6cfe7a10a534 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:25.098253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc90f47-8a41-4517-bf12-6cfe7a10a534 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:25.106500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:25.118897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584b7c9-5414-4489-91c5-7d76f77881a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-6cc90f47-8a41-4517-bf12-6cfe7a10a534 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:25.119734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:25.120817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:29:25.121946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:25.122605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584b7c9-5414-4489-91c5-7d76f77881a9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:25.122605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584b7c9-5414-4489-91c5-7d76f77881a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:25.123046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584b7c9-5414-4489-91c5-7d76f77881a9 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:25.124039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584b7c9-5414-4489-91c5-7d76f77881a9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:25.128918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584b7c9-5414-4489-91c5-7d76f77881a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:25.128918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584b7c9-5414-4489-91c5-7d76f77881a9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:25.129223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 489/971] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:29:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:25.130006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc90f47-8a41-4517-bf12-6cfe7a10a534 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:25.175963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 483/972] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 13 20:29:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:25.244673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc90f47-8a41-4517-bf12-6cfe7a10a534 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:25.284002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92001b-1fb5-40b7-897b-9802533bd573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:25.285097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92001b-1fb5-40b7-897b-9802533bd573 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:25.285596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92001b-1fb5-40b7-897b-9802533bd573 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:25.286085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92001b-1fb5-40b7-897b-9802533bd573 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:25.287178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92001b-1fb5-40b7-897b-9802533bd573 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:25.287788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92001b-1fb5-40b7-897b-9802533bd573 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:25.289543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92001b-1fb5-40b7-897b-9802533bd573 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:25.310421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 490/973] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 13 20:29:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:25.316482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584b7c9-5414-4489-91c5-7d76f77881a9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:25.418991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584b7c9-5414-4489-91c5-7d76f77881a9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:25.453774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5400c0f-c68a-4816-98be-c6c7a777472c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:25.454426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5400c0f-c68a-4816-98be-c6c7a777472c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:25.454707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5400c0f-c68a-4816-98be-c6c7a777472c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:25.454986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5400c0f-c68a-4816-98be-c6c7a777472c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:25.502863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:25.521930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5400c0f-c68a-4816-98be-c6c7a777472c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:25.523398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5400c0f-c68a-4816-98be-c6c7a777472c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:25.627080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 484/974] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:25 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:25.632541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92001b-1fb5-40b7-897b-9802533bd573 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:25.678637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:25.732874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92001b-1fb5-40b7-897b-9802533bd573 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:25.766849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87a936-aad8-44c5-95f4-819b9b0a8d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:25.767614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87a936-aad8-44c5-95f4-819b9b0a8d6c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:25.767896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87a936-aad8-44c5-95f4-819b9b0a8d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:25.768168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87a936-aad8-44c5-95f4-819b9b0a8d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:25.778174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87a936-aad8-44c5-95f4-819b9b0a8d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:25.779642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87a936-aad8-44c5-95f4-819b9b0a8d6c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7f87a936-aad8-44c5-95f4-819b9b0a8d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:25.798074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:25.798976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:25.800332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:25.800332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:25.800332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:25.800332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:25.800868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 491/975] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:25.804895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5400c0f-c68a-4816-98be-c6c7a777472c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:25.887595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:25.907430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5400c0f-c68a-4816-98be-c6c7a777472c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:25.944963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27187b6-f1a9-4f02-9c7c-b4ce452ef348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:25.945628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27187b6-f1a9-4f02-9c7c-b4ce452ef348 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:25.945926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27187b6-f1a9-4f02-9c7c-b4ce452ef348 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:25.946187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27187b6-f1a9-4f02-9c7c-b4ce452ef348 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:25.946958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27187b6-f1a9-4f02-9c7c-b4ce452ef348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:25.947447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27187b6-f1a9-4f02-9c7c-b4ce452ef348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:25.950267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27187b6-f1a9-4f02-9c7c-b4ce452ef348 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:25.963134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 485/976] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:25.968541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87a936-aad8-44c5-95f4-819b9b0a8d6c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:26.066986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:26.073340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87a936-aad8-44c5-95f4-819b9b0a8d6c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:26.107524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18401ab0-2a3f-4956-940a-52dd3746caa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:26.108341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18401ab0-2a3f-4956-940a-52dd3746caa4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.108633 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18401ab0-2a3f-4956-940a-52dd3746caa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.108901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18401ab0-2a3f-4956-940a-52dd3746caa4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.116114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18401ab0-2a3f-4956-940a-52dd3746caa4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:26.117556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18401ab0-2a3f-4956-940a-52dd3746caa4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-18401ab0-2a3f-4956-940a-52dd3746caa4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:26.131361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:26.132387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:26.133362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:26.133362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:26.133362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:26.133362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:26.133362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:26.133362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:26.133362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:26.133362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:26.134342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 492/977] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:26.138614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27187b6-f1a9-4f02-9c7c-b4ce452ef348 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:26.206305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf6405-6d62-48ee-a6e4-37b9cda77c1b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.206637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf6405-6d62-48ee-a6e4-37b9cda77c1b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.206865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf6405-6d62-48ee-a6e4-37b9cda77c1b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.209108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 493/978] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:26.213901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f67be5-d67e-4a0b-98e3-f787ffc6df74 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.214191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f67be5-d67e-4a0b-98e3-f787ffc6df74 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.214470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f67be5-d67e-4a0b-98e3-f787ffc6df74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.231773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-29f67be5-d67e-4a0b-98e3-f787ffc6df74 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:26.232976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-29f67be5-d67e-4a0b-98e3-f787ffc6df74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:26.242058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27187b6-f1a9-4f02-9c7c-b4ce452ef348 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:26.276191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f276d91-b6c7-47f2-b813-2a70194f14b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:26.276852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276d91-b6c7-47f2-b813-2a70194f14b4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.277135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276d91-b6c7-47f2-b813-2a70194f14b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.277407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276d91-b6c7-47f2-b813-2a70194f14b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.278184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f276d91-b6c7-47f2-b813-2a70194f14b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:26.278490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f276d91-b6c7-47f2-b813-2a70194f14b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:26.279950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f276d91-b6c7-47f2-b813-2a70194f14b4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:26.297006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 486/979] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:26.298132 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f276d91-b6c7-47f2-b813-2a70194f14b4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:26.300983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:26.308649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 494/980] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:26.331059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f67be5-d67e-4a0b-98e3-f787ffc6df74 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:26.405416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f276d91-b6c7-47f2-b813-2a70194f14b4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:26.420546 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:26.437343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f67be5-d67e-4a0b-98e3-f787ffc6df74 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:26.440521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6beefe-c5f8-428e-ab32-6991600bbe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:26.441175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6beefe-c5f8-428e-ab32-6991600bbe3b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.441459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6beefe-c5f8-428e-ab32-6991600bbe3b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.441737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6beefe-c5f8-428e-ab32-6991600bbe3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.442568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6beefe-c5f8-428e-ab32-6991600bbe3b None admin] RBAC: Adding query filter params (role.id=3c7426b3b0754e6089b5ccc02c8b3eec, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=95dcfd3d412a47298d7d5cf7b3b56f97) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:26.442884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6beefe-c5f8-428e-ab32-6991600bbe3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:26.444353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6beefe-c5f8-428e-ab32-6991600bbe3b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:26.450945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 487/981] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:26 2022] GET /identity/v3/role_assignments?user.id=95dcfd3d412a47298d7d5cf7b3b56f97&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=3c7426b3b0754e6089b5ccc02c8b3eec => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:26.474122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648b6e3-5d98-4bc6-a46e-726fb4f1c509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:26.474814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648b6e3-5d98-4bc6-a46e-726fb4f1c509 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.475099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648b6e3-5d98-4bc6-a46e-726fb4f1c509 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.475375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648b6e3-5d98-4bc6-a46e-726fb4f1c509 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.476143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648b6e3-5d98-4bc6-a46e-726fb4f1c509 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:26.477580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648b6e3-5d98-4bc6-a46e-726fb4f1c509 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:26.494630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 495/982] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:29:26 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:29:26.497208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce45b84-da9e-4318-a9d5-e0027d685962 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.497502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce45b84-da9e-4318-a9d5-e0027d685962 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.497805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce45b84-da9e-4318-a9d5-e0027d685962 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.499171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 488/983] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:26.505426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f00a-1438-4fcb-9220-f3901d24d8b2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.505769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f00a-1438-4fcb-9220-f3901d24d8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.506064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f00a-1438-4fcb-9220-f3901d24d8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.525291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-70e9f00a-1438-4fcb-9220-f3901d24d8b2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:26.526572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-70e9f00a-1438-4fcb-9220-f3901d24d8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:26.619573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 496/984] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:26.641478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce45b84-da9e-4318-a9d5-e0027d685962 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:26.712005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:26.748828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce45b84-da9e-4318-a9d5-e0027d685962 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:26.784759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3aa5c-9d19-4496-9b61-26a8c3739eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:26.785572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3aa5c-9d19-4496-9b61-26a8c3739eea None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.785906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3aa5c-9d19-4496-9b61-26a8c3739eea None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.786200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3aa5c-9d19-4496-9b61-26a8c3739eea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.786916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3aa5c-9d19-4496-9b61-26a8c3739eea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:26.788370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3aa5c-9d19-4496-9b61-26a8c3739eea None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7ba3aa5c-9d19-4496-9b61-26a8c3739eea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:26.800904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:26.802079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:26.803122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:26.804127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 489/985] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:26.808558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f00a-1438-4fcb-9220-f3901d24d8b2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:26.881799 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:26.914610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f00a-1438-4fcb-9220-f3901d24d8b2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:26.949344 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d6254-fcad-41aa-b0d4-ec85d0ca9d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:26.950182 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d6254-fcad-41aa-b0d4-ec85d0ca9d0d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:26.950571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d6254-fcad-41aa-b0d4-ec85d0ca9d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:26.950945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d6254-fcad-41aa-b0d4-ec85d0ca9d0d None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:26.951786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d6254-fcad-41aa-b0d4-ec85d0ca9d0d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:26.952213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d6254-fcad-41aa-b0d4-ec85d0ca9d0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:26.953687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d6254-fcad-41aa-b0d4-ec85d0ca9d0d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:26.960524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 497/986] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:26.966319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3aa5c-9d19-4496-9b61-26a8c3739eea None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:27.076523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3aa5c-9d19-4496-9b61-26a8c3739eea None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:27.106732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:27.113684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450e3f-c4ab-4866-bc1a-22559d2c0723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:27.114414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450e3f-c4ab-4866-bc1a-22559d2c0723 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:27.114751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450e3f-c4ab-4866-bc1a-22559d2c0723 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:27.115039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450e3f-c4ab-4866-bc1a-22559d2c0723 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:27.115796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450e3f-c4ab-4866-bc1a-22559d2c0723 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:27.116112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450e3f-c4ab-4866-bc1a-22559d2c0723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:27.117498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450e3f-c4ab-4866-bc1a-22559d2c0723 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:27.127386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 490/987] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:27.129038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d6254-fcad-41aa-b0d4-ec85d0ca9d0d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:27.229878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d6254-fcad-41aa-b0d4-ec85d0ca9d0d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:27.263426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4f5f1-e5a9-4bfb-8575-bc39285be0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:27.264113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4f5f1-e5a9-4bfb-8575-bc39285be0fd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:27.264399 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4f5f1-e5a9-4bfb-8575-bc39285be0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:27.264678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4f5f1-e5a9-4bfb-8575-bc39285be0fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:27.265511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4f5f1-e5a9-4bfb-8575-bc39285be0fd None admin] RBAC: Adding query filter params (interface=admin, service_id=d04275f94c404cce91ee16654a02c891, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:27.265830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4f5f1-e5a9-4bfb-8575-bc39285be0fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:27.267203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4f5f1-e5a9-4bfb-8575-bc39285be0fd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:27.269582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:27.274268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 498/988] 10.222.0.47 () {56 vars in 1249 bytes} [Wed Jul 13 20:29:27 2022] GET /identity/v3/endpoints?service_id=d04275f94c404cce91ee16654a02c891&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:27.280071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450e3f-c4ab-4866-bc1a-22559d2c0723 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:27.382258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450e3f-c4ab-4866-bc1a-22559d2c0723 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:27.414833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf83483-cb70-4d67-b105-9c267fc6de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:27.415514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf83483-cb70-4d67-b105-9c267fc6de3d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:27.415808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf83483-cb70-4d67-b105-9c267fc6de3d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:27.416114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf83483-cb70-4d67-b105-9c267fc6de3d None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:27.416791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf83483-cb70-4d67-b105-9c267fc6de3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:27.418171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf83483-cb70-4d67-b105-9c267fc6de3d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:27.424757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 491/989] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 13 20:29:27 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:29:28.474983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42c9a1-7854-4ead-a393-9e86182a29cb None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:28.475536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42c9a1-7854-4ead-a393-9e86182a29cb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:28.475536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42c9a1-7854-4ead-a393-9e86182a29cb None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:28.479497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 499/990] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:28.482340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce018cb9-6c09-4894-a030-ca6bf6b09b5d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:28.482733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce018cb9-6c09-4894-a030-ca6bf6b09b5d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:28.482893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce018cb9-6c09-4894-a030-ca6bf6b09b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:28.500338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ce018cb9-6c09-4894-a030-ca6bf6b09b5d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:28.501872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ce018cb9-6c09-4894-a030-ca6bf6b09b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:28.549950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351f522-6ebe-42dc-8548-fe64249ea2c4 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:28.550187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351f522-6ebe-42dc-8548-fe64249ea2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:28.550502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351f522-6ebe-42dc-8548-fe64249ea2c4 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:28.552054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 500/991] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:28.556827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c969c94-47ad-43e2-9fa9-1d5eec740610 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:28.557088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c969c94-47ad-43e2-9fa9-1d5eec740610 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:28.557358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c969c94-47ad-43e2-9fa9-1d5eec740610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:28.562899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:28.576417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0c969c94-47ad-43e2-9fa9-1d5eec740610 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:28.577622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0c969c94-47ad-43e2-9fa9-1d5eec740610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:28.580553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 492/992] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:28.602136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce018cb9-6c09-4894-a030-ca6bf6b09b5d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:28.650225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 501/993] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:28.672486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c969c94-47ad-43e2-9fa9-1d5eec740610 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:28.700742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce018cb9-6c09-4894-a030-ca6bf6b09b5d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:28.733336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcab64b-206e-4fea-be8c-6a8ad0b3f971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:28.734052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcab64b-206e-4fea-be8c-6a8ad0b3f971 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:28.734332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcab64b-206e-4fea-be8c-6a8ad0b3f971 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:28.734668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcab64b-206e-4fea-be8c-6a8ad0b3f971 None admin] PATH_INFO: `/v3/services/image` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:28.735318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcab64b-206e-4fea-be8c-6a8ad0b3f971 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:28.736744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcab64b-206e-4fea-be8c-6a8ad0b3f971 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2bcab64b-206e-4fea-be8c-6a8ad0b3f971 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:28.748371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:28.749367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:29:28.750218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:28.754486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 493/994] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Jul 13 20:29:28 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:28.755286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcab64b-206e-4fea-be8c-6a8ad0b3f971 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:28.776712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c969c94-47ad-43e2-9fa9-1d5eec740610 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:28.805490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:28.806672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:28.813586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa820830-51c3-447f-bf2c-080ead11a59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:28.814394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa820830-51c3-447f-bf2c-080ead11a59e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:28.814685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa820830-51c3-447f-bf2c-080ead11a59e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:28.814982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa820830-51c3-447f-bf2c-080ead11a59e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:28.815641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa820830-51c3-447f-bf2c-080ead11a59e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:28.817037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa820830-51c3-447f-bf2c-080ead11a59e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fa820830-51c3-447f-bf2c-080ead11a59e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:28.828930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:28.829890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:28.830723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 502/995] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:28.835992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa820830-51c3-447f-bf2c-080ead11a59e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:28.863475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcab64b-206e-4fea-be8c-6a8ad0b3f971 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:28.896900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12093a-3cf4-41c9-a591-5d7de29e0817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:28.897570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12093a-3cf4-41c9-a591-5d7de29e0817 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:28.897854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12093a-3cf4-41c9-a591-5d7de29e0817 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:28.898129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12093a-3cf4-41c9-a591-5d7de29e0817 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:28.898826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12093a-3cf4-41c9-a591-5d7de29e0817 None admin] RBAC: Adding query filter params (name=image) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:28.899139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12093a-3cf4-41c9-a591-5d7de29e0817 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:28.900628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12093a-3cf4-41c9-a591-5d7de29e0817 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:28.905936 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 494/996] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:28.911239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12093a-3cf4-41c9-a591-5d7de29e0817 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:28.938973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa820830-51c3-447f-bf2c-080ead11a59e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:28.974581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c3d44-7a6a-4831-951a-ca2891b8cf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:28.975247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c3d44-7a6a-4831-951a-ca2891b8cf6f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:28.975593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c3d44-7a6a-4831-951a-ca2891b8cf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:28.975855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c3d44-7a6a-4831-951a-ca2891b8cf6f None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:28.976594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c3d44-7a6a-4831-951a-ca2891b8cf6f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:28.976911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c3d44-7a6a-4831-951a-ca2891b8cf6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:28.978293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c3d44-7a6a-4831-951a-ca2891b8cf6f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:28.983254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 503/997] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:28.988746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c3d44-7a6a-4831-951a-ca2891b8cf6f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:29.017711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12093a-3cf4-41c9-a591-5d7de29e0817 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:29.034815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:29.055743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:29.057787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98bd496-0d18-4ef1-afb7-31aadf8185f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:29.058709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98bd496-0d18-4ef1-afb7-31aadf8185f5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:29.059228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98bd496-0d18-4ef1-afb7-31aadf8185f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:29.059417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98bd496-0d18-4ef1-afb7-31aadf8185f5 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:29.062969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98bd496-0d18-4ef1-afb7-31aadf8185f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:29.062969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98bd496-0d18-4ef1-afb7-31aadf8185f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:29.062969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98bd496-0d18-4ef1-afb7-31aadf8185f5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:29.068430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 495/998] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:29.102511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c3d44-7a6a-4831-951a-ca2891b8cf6f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:29.140902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21652b11-50f8-4467-8e5c-38aa9ea5262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:29.141599 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21652b11-50f8-4467-8e5c-38aa9ea5262b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:29.141885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21652b11-50f8-4467-8e5c-38aa9ea5262b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:29.142166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21652b11-50f8-4467-8e5c-38aa9ea5262b None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:29.142890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21652b11-50f8-4467-8e5c-38aa9ea5262b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:29.143202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21652b11-50f8-4467-8e5c-38aa9ea5262b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:29.144641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21652b11-50f8-4467-8e5c-38aa9ea5262b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:29.155450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 504/999] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:29.156673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98bd496-0d18-4ef1-afb7-31aadf8185f5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:29.268679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98bd496-0d18-4ef1-afb7-31aadf8185f5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:29.302472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcb74a-7833-475b-aa5e-e9009644c70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:29.303401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcb74a-7833-475b-aa5e-e9009644c70a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:29.303843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcb74a-7833-475b-aa5e-e9009644c70a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:29.304282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcb74a-7833-475b-aa5e-e9009644c70a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:29.305221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcb74a-7833-475b-aa5e-e9009644c70a None admin] RBAC: Adding query filter params (interface=public, service_id=559e37a95e33400fb0e0dc276b51cc25, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:29.305671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcb74a-7833-475b-aa5e-e9009644c70a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:29.307237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcb74a-7833-475b-aa5e-e9009644c70a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:29.313034 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 496/1000] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 13 20:29:29 2022] GET /identity/v3/endpoints?service_id=559e37a95e33400fb0e0dc276b51cc25&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:29.318816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21652b11-50f8-4467-8e5c-38aa9ea5262b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:29.423927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21652b11-50f8-4467-8e5c-38aa9ea5262b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:29.447045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec507a5d-1df5-4ecc-b758-7d32491cff0b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:29.447800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:29.448045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec507a5d-1df5-4ecc-b758-7d32491cff0b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:29.448335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec507a5d-1df5-4ecc-b758-7d32491cff0b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:29.450044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 497/1001] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:29.455228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58529-c287-417e-ad2a-f233e3bdb76b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:29.455507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58529-c287-417e-ad2a-f233e3bdb76b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:29.455824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58529-c287-417e-ad2a-f233e3bdb76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:29.463124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234511-c9e8-465c-bb97-a39dcc368070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:29.463834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234511-c9e8-465c-bb97-a39dcc368070 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:29.464122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234511-c9e8-465c-bb97-a39dcc368070 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:29.464398 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234511-c9e8-465c-bb97-a39dcc368070 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:29.465078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65234511-c9e8-465c-bb97-a39dcc368070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:29.466457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65234511-c9e8-465c-bb97-a39dcc368070 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:29.473106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5ce58529-c287-417e-ad2a-f233e3bdb76b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:29.479488 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 505/1002] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 13 20:29:29 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:29:29.480008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5ce58529-c287-417e-ad2a-f233e3bdb76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:29.524607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:29.555127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 498/1003] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:29.578828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234511-c9e8-465c-bb97-a39dcc368070 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:29.689010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234511-c9e8-465c-bb97-a39dcc368070 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:29.724798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502e62f-b45d-4d38-bc89-c1b5bb0f19d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:29.725591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502e62f-b45d-4d38-bc89-c1b5bb0f19d6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:29.725849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502e62f-b45d-4d38-bc89-c1b5bb0f19d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:29.726139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502e62f-b45d-4d38-bc89-c1b5bb0f19d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:29.726786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502e62f-b45d-4d38-bc89-c1b5bb0f19d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:29.728186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502e62f-b45d-4d38-bc89-c1b5bb0f19d6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-b502e62f-b45d-4d38-bc89-c1b5bb0f19d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:29.740317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:29.741457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:29.744526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 506/1004] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 13 20:29:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:29.750620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58529-c287-417e-ad2a-f233e3bdb76b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:29.857902 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58529-c287-417e-ad2a-f233e3bdb76b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:29.891442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae5814-8c55-44be-b2d3-6def8d74437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:29.892143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5814-8c55-44be-b2d3-6def8d74437d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:29.892430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5814-8c55-44be-b2d3-6def8d74437d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:29.892712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5814-8c55-44be-b2d3-6def8d74437d None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:29.893413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae5814-8c55-44be-b2d3-6def8d74437d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:29.893721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae5814-8c55-44be-b2d3-6def8d74437d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:29.895107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae5814-8c55-44be-b2d3-6def8d74437d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:29.900050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 499/1005] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:29.905964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502e62f-b45d-4d38-bc89-c1b5bb0f19d6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:29.957859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:30.008680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502e62f-b45d-4d38-bc89-c1b5bb0f19d6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:30.042866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636173b1-19a5-4ada-a971-0ca6df37a1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:30.043562 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636173b1-19a5-4ada-a971-0ca6df37a1df None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:30.043852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636173b1-19a5-4ada-a971-0ca6df37a1df None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:30.044133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636173b1-19a5-4ada-a971-0ca6df37a1df None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:30.044844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636173b1-19a5-4ada-a971-0ca6df37a1df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:30.045155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636173b1-19a5-4ada-a971-0ca6df37a1df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:30.046532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636173b1-19a5-4ada-a971-0ca6df37a1df None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:30.056139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 507/1006] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 13 20:29:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:30.062101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae5814-8c55-44be-b2d3-6def8d74437d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:30.103344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:30.164997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae5814-8c55-44be-b2d3-6def8d74437d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:30.200669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf58aaf-5346-4f8d-8eef-20c43c453667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:30.201342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf58aaf-5346-4f8d-8eef-20c43c453667 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:30.201628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf58aaf-5346-4f8d-8eef-20c43c453667 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:30.201908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf58aaf-5346-4f8d-8eef-20c43c453667 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:30.202664 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf58aaf-5346-4f8d-8eef-20c43c453667 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:30.204086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf58aaf-5346-4f8d-8eef-20c43c453667 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:30.227492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 500/1007] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 13 20:29:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:29:31.078572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b848cb-169e-42bf-aa39-cb323f61e50b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:31.079032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b848cb-169e-42bf-aa39-cb323f61e50b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:31.079107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b848cb-169e-42bf-aa39-cb323f61e50b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:31.080515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 508/1008] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:31.085883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bb05f-cb0e-4651-9d8f-0ad01131e39a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:31.086323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bb05f-cb0e-4651-9d8f-0ad01131e39a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:31.086469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bb05f-cb0e-4651-9d8f-0ad01131e39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:31.103983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e36bb05f-cb0e-4651-9d8f-0ad01131e39a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:31.105168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e36bb05f-cb0e-4651-9d8f-0ad01131e39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:31.179508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 501/1009] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:31 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:31.202174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b848cb-169e-42bf-aa39-cb323f61e50b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:31.308275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b848cb-169e-42bf-aa39-cb323f61e50b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:31.324196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:31.346404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c62666-d279-4577-90f8-22d0743a9ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:31.347076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c62666-d279-4577-90f8-22d0743a9ee4 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:31.347361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c62666-d279-4577-90f8-22d0743a9ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:31.347664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c62666-d279-4577-90f8-22d0743a9ee4 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:31.348397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c62666-d279-4577-90f8-22d0743a9ee4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:31.349774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c62666-d279-4577-90f8-22d0743a9ee4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:31.364367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 509/1010] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:29:31 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:31.480696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210baa-9d72-4ae5-afa5-e4cf67a953f4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:31.481117 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210baa-9d72-4ae5-afa5-e4cf67a953f4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:31.481293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210baa-9d72-4ae5-afa5-e4cf67a953f4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:31.483491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 502/1011] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:31.488528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9ac79-0245-4925-a5bd-1898c64724b7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:31.488845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9ac79-0245-4925-a5bd-1898c64724b7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:31.489127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9ac79-0245-4925-a5bd-1898c64724b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:31.507043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f6b9ac79-0245-4925-a5bd-1898c64724b7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:31.508319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f6b9ac79-0245-4925-a5bd-1898c64724b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:31.582286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 510/1012] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:31 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:31.605396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210baa-9d72-4ae5-afa5-e4cf67a953f4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:31.634851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:31.717388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210baa-9d72-4ae5-afa5-e4cf67a953f4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:31.752346 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a0a4f-1139-4916-81a1-ea5abd801602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:31.753146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a0a4f-1139-4916-81a1-ea5abd801602 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:31.753435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a0a4f-1139-4916-81a1-ea5abd801602 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:31.753684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a0a4f-1139-4916-81a1-ea5abd801602 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:31.754355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a0a4f-1139-4916-81a1-ea5abd801602 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:31.755751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a0a4f-1139-4916-81a1-ea5abd801602 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-325a0a4f-1139-4916-81a1-ea5abd801602 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:31.768354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:31.769379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:31.770197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 503/1013] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:31.776764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9ac79-0245-4925-a5bd-1898c64724b7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:31.878348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:31.881968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9ac79-0245-4925-a5bd-1898c64724b7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:31.918811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8b7c1-fd16-4387-96de-d53494320281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:31.919502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8b7c1-fd16-4387-96de-d53494320281 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:31.919802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8b7c1-fd16-4387-96de-d53494320281 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:31.920083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8b7c1-fd16-4387-96de-d53494320281 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:31.920794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8b7c1-fd16-4387-96de-d53494320281 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:31.921105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8b7c1-fd16-4387-96de-d53494320281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:31.922473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8b7c1-fd16-4387-96de-d53494320281 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:31.927648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 511/1014] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:31.933514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a0a4f-1139-4916-81a1-ea5abd801602 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:32.022683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:32.036487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a0a4f-1139-4916-81a1-ea5abd801602 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:32.079201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3df398-4ec3-404a-bf3e-321510a27b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:32.079908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3df398-4ec3-404a-bf3e-321510a27b40 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:32.080201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3df398-4ec3-404a-bf3e-321510a27b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:32.080485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3df398-4ec3-404a-bf3e-321510a27b40 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:32.081193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3df398-4ec3-404a-bf3e-321510a27b40 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:32.081505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3df398-4ec3-404a-bf3e-321510a27b40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:32.082918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3df398-4ec3-404a-bf3e-321510a27b40 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:32.088107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 504/1015] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 13 20:29:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:32.093776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8b7c1-fd16-4387-96de-d53494320281 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:32.193541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8b7c1-fd16-4387-96de-d53494320281 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:32.228013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5880121-a369-4e30-9cb4-9d90304490d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:32.228728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5880121-a369-4e30-9cb4-9d90304490d6 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:32.229014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5880121-a369-4e30-9cb4-9d90304490d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:32.229291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5880121-a369-4e30-9cb4-9d90304490d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:32.230049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5880121-a369-4e30-9cb4-9d90304490d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:32.231420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5880121-a369-4e30-9cb4-9d90304490d6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:32.250083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 512/1016] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 13 20:29:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:29:32.262952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7873c6a-decb-44e4-be6f-830522ac11ae None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:32.263283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7873c6a-decb-44e4-be6f-830522ac11ae None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:32.263551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7873c6a-decb-44e4-be6f-830522ac11ae None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:32.265296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 505/1017] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:32.270934 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ed43-4009-4018-ae89-021e0a7866dc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:32.271226 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ed43-4009-4018-ae89-021e0a7866dc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:32.271539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ed43-4009-4018-ae89-021e0a7866dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:32.289646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a140ed43-4009-4018-ae89-021e0a7866dc None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:32.291006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a140ed43-4009-4018-ae89-021e0a7866dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:32.308082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:32.376048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 513/1018] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:32.399773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7873c6a-decb-44e4-be6f-830522ac11ae None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:32.511095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7873c6a-decb-44e4-be6f-830522ac11ae None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:32.549913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7683c-9d6e-4a12-be4d-61b335b395cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:32.550730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7683c-9d6e-4a12-be4d-61b335b395cf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:32.551010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7683c-9d6e-4a12-be4d-61b335b395cf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:32.551280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7683c-9d6e-4a12-be4d-61b335b395cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:32.552591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7683c-9d6e-4a12-be4d-61b335b395cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:32.554056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7683c-9d6e-4a12-be4d-61b335b395cf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:32.555948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 506/1019] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:32.561692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ed43-4009-4018-ae89-021e0a7866dc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:32.665028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ed43-4009-4018-ae89-021e0a7866dc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:32.700743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ae08a-772e-42a8-9c63-fd83f9d3e8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:32.701403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ae08a-772e-42a8-9c63-fd83f9d3e8b3 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:32.701687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ae08a-772e-42a8-9c63-fd83f9d3e8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:32.701952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ae08a-772e-42a8-9c63-fd83f9d3e8b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:32.702665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ae08a-772e-42a8-9c63-fd83f9d3e8b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:32.704130 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ae08a-772e-42a8-9c63-fd83f9d3e8b3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:32.724372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:32.733857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 514/1020] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:29:32 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:33.385353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f64b03-c793-4ee9-ae3a-8c12efc8d413 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:33.385861 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f64b03-c793-4ee9-ae3a-8c12efc8d413 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:33.385861 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f64b03-c793-4ee9-ae3a-8c12efc8d413 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:33.387350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 507/1021] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:33.394269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea00c-3be5-4645-bcd0-9b16d946ba22 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:33.394786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea00c-3be5-4645-bcd0-9b16d946ba22 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:33.394786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea00c-3be5-4645-bcd0-9b16d946ba22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:33.412776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dafea00c-3be5-4645-bcd0-9b16d946ba22 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:33.413968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dafea00c-3be5-4645-bcd0-9b16d946ba22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:33.488506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 515/1022] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:33.510987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f64b03-c793-4ee9-ae3a-8c12efc8d413 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:33.534678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:33.617116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f64b03-c793-4ee9-ae3a-8c12efc8d413 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:33.654695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457399-2714-4195-9bbf-f5f43696914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:33.655451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457399-2714-4195-9bbf-f5f43696914e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:33.655827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457399-2714-4195-9bbf-f5f43696914e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:33.656075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457399-2714-4195-9bbf-f5f43696914e None admin] PATH_INFO: `/v3/services/image` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:33.656749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0457399-2714-4195-9bbf-f5f43696914e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:33.658105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0457399-2714-4195-9bbf-f5f43696914e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c0457399-2714-4195-9bbf-f5f43696914e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:33.669590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:33.670578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:33.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 508/1023] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Jul 13 20:29:33 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:33.677120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafea00c-3be5-4645-bcd0-9b16d946ba22 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:33.781698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafea00c-3be5-4645-bcd0-9b16d946ba22 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:33.810084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:33.818734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bab3e2-7a27-49fa-b566-257076f85b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:33.819409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab3e2-7a27-49fa-b566-257076f85b2e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:33.819720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab3e2-7a27-49fa-b566-257076f85b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:33.820002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab3e2-7a27-49fa-b566-257076f85b2e None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:33.820712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bab3e2-7a27-49fa-b566-257076f85b2e None admin] RBAC: Adding query filter params (name=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:33.821022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bab3e2-7a27-49fa-b566-257076f85b2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:33.822783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bab3e2-7a27-49fa-b566-257076f85b2e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:33.831491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 516/1024] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:33.833746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457399-2714-4195-9bbf-f5f43696914e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:33.922008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:33.940535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457399-2714-4195-9bbf-f5f43696914e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:33.975819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72846c99-4373-44e1-a6a4-7e8cc4bc8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:33.976750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72846c99-4373-44e1-a6a4-7e8cc4bc8f37 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:33.977198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72846c99-4373-44e1-a6a4-7e8cc4bc8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:33.977620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72846c99-4373-44e1-a6a4-7e8cc4bc8f37 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:33.978559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72846c99-4373-44e1-a6a4-7e8cc4bc8f37 None admin] RBAC: Adding query filter params (type=image) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:33.979011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72846c99-4373-44e1-a6a4-7e8cc4bc8f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:33.980569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72846c99-4373-44e1-a6a4-7e8cc4bc8f37 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:33.986267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 509/1025] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:33.991962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bab3e2-7a27-49fa-b566-257076f85b2e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:34.105689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bab3e2-7a27-49fa-b566-257076f85b2e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:34.141208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ce7a6-c047-4198-bd94-326cd233c1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:34.141871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ce7a6-c047-4198-bd94-326cd233c1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:34.142163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ce7a6-c047-4198-bd94-326cd233c1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:34.142440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ce7a6-c047-4198-bd94-326cd233c1c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:34.143239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ce7a6-c047-4198-bd94-326cd233c1c1 None admin] RBAC: Adding query filter params (interface=public, service_id=35d4116831914ab1b2318c32aaf303ae, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:34.143581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ce7a6-c047-4198-bd94-326cd233c1c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:34.144961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ce7a6-c047-4198-bd94-326cd233c1c1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:34.150326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 517/1026] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 13 20:29:33 2022] GET /identity/v3/endpoints?service_id=35d4116831914ab1b2318c32aaf303ae&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:34.155778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72846c99-4373-44e1-a6a4-7e8cc4bc8f37 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:34.255278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72846c99-4373-44e1-a6a4-7e8cc4bc8f37 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:34.288632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65def8-8dd1-4c7d-aa8b-f7fe14022f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:34.289304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65def8-8dd1-4c7d-aa8b-f7fe14022f77 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:34.289590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65def8-8dd1-4c7d-aa8b-f7fe14022f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:34.289871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65def8-8dd1-4c7d-aa8b-f7fe14022f77 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:34.290548 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65def8-8dd1-4c7d-aa8b-f7fe14022f77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:34.292010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65def8-8dd1-4c7d-aa8b-f7fe14022f77 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:34.298966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 510/1027] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 13 20:29:34 2022] GET /identity/v3/services => generated 1957 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:29:34.369223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c999d-7143-4754-b4fd-bc4dd9e99469 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:34.369602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c999d-7143-4754-b4fd-bc4dd9e99469 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:34.369951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c999d-7143-4754-b4fd-bc4dd9e99469 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:34.371726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 518/1028] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:34.377610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4939d2-fb6e-4b15-80c0-7d350554af00 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:34.377922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4939d2-fb6e-4b15-80c0-7d350554af00 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:34.378167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4939d2-fb6e-4b15-80c0-7d350554af00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:34.396306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4a4939d2-fb6e-4b15-80c0-7d350554af00 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:34.397504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4a4939d2-fb6e-4b15-80c0-7d350554af00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:34.406973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:34.475783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 511/1029] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:34.499216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c999d-7143-4754-b4fd-bc4dd9e99469 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:34.560154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:34.608829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c999d-7143-4754-b4fd-bc4dd9e99469 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:34.643187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96be95-5f3c-4e23-937f-2a87b36883b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:34.644035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96be95-5f3c-4e23-937f-2a87b36883b8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:34.644314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96be95-5f3c-4e23-937f-2a87b36883b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:34.644587 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96be95-5f3c-4e23-937f-2a87b36883b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:34.654642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96be95-5f3c-4e23-937f-2a87b36883b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0c96be95-5f3c-4e23-937f-2a87b36883b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:34.662834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:34.663837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:34.664675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:34.667545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 519/1030] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 13 20:29:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:34.670468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4939d2-fb6e-4b15-80c0-7d350554af00 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:34.780166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442c198-add0-4954-b949-ad683611df79 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:34.780471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442c198-add0-4954-b949-ad683611df79 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:34.780716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442c198-add0-4954-b949-ad683611df79 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:34.782300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 520/1031] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:34.787409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d8f2-7fcf-4f15-afad-218ff9317f35 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:34.787744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d8f2-7fcf-4f15-afad-218ff9317f35 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:34.788028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d8f2-7fcf-4f15-afad-218ff9317f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:34.793531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4939d2-fb6e-4b15-80c0-7d350554af00 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:34.805251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-aa23d8f2-7fcf-4f15-afad-218ff9317f35 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:34.806474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aa23d8f2-7fcf-4f15-afad-218ff9317f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:34.830707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744222ac-7647-4591-ba6b-56c952b3e29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:34.831308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744222ac-7647-4591-ba6b-56c952b3e29a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:34.831623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744222ac-7647-4591-ba6b-56c952b3e29a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:34.831903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744222ac-7647-4591-ba6b-56c952b3e29a None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:34.832648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744222ac-7647-4591-ba6b-56c952b3e29a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:34.832957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744222ac-7647-4591-ba6b-56c952b3e29a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:34.834336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744222ac-7647-4591-ba6b-56c952b3e29a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:34.847495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 512/1032] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 13 20:29:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:34.848516 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744222ac-7647-4591-ba6b-56c952b3e29a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:34.852919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:34.876836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 521/1033] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:34.900769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23d8f2-7fcf-4f15-afad-218ff9317f35 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:34.950679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:34.954710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744222ac-7647-4591-ba6b-56c952b3e29a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:34.989635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b25b15-e03a-42da-9cc7-158b9380dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:34.990482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b25b15-e03a-42da-9cc7-158b9380dcce None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:34.990732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b25b15-e03a-42da-9cc7-158b9380dcce None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:34.991012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b25b15-e03a-42da-9cc7-158b9380dcce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:35.002020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b25b15-e03a-42da-9cc7-158b9380dcce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:35.003832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b25b15-e03a-42da-9cc7-158b9380dcce None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:35.017156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23d8f2-7fcf-4f15-afad-218ff9317f35 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a9b25b15-e03a-42da-9cc7-158b9380dcce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:35.021821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:35.022704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:35.029298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:35.029298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:35.029298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:35.029298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:35.029298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 513/1034] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:35.029298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b25b15-e03a-42da-9cc7-158b9380dcce None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:35.059144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466feb32-996a-4669-a396-53238a18d0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:35.060080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466feb32-996a-4669-a396-53238a18d0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:35.060476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466feb32-996a-4669-a396-53238a18d0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:35.060754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466feb32-996a-4669-a396-53238a18d0f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:35.061920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466feb32-996a-4669-a396-53238a18d0f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:35.063601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466feb32-996a-4669-a396-53238a18d0f1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:35.066677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 522/1035] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:35.070938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466feb32-996a-4669-a396-53238a18d0f1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:35.102536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:35.150252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b25b15-e03a-42da-9cc7-158b9380dcce None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:35.184462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9f857-7e69-448a-903b-ec6c05657bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:35.185125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9f857-7e69-448a-903b-ec6c05657bbd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:35.185412 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9f857-7e69-448a-903b-ec6c05657bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:35.185576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466feb32-996a-4669-a396-53238a18d0f1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:35.185678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9f857-7e69-448a-903b-ec6c05657bbd None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:35.186396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9f857-7e69-448a-903b-ec6c05657bbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:35.186704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9f857-7e69-448a-903b-ec6c05657bbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:35.188129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9f857-7e69-448a-903b-ec6c05657bbd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:35.203490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 514/1036] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:35.203673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:35.206225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9f857-7e69-448a-903b-ec6c05657bbd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:35.225136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fda0f-04eb-43e2-9c1b-67e98e5e1b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:35.225821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fda0f-04eb-43e2-9c1b-67e98e5e1b20 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:35.226131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fda0f-04eb-43e2-9c1b-67e98e5e1b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:35.226670 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fda0f-04eb-43e2-9c1b-67e98e5e1b20 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:35.227391 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fda0f-04eb-43e2-9c1b-67e98e5e1b20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:35.228949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fda0f-04eb-43e2-9c1b-67e98e5e1b20 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:35.251173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 523/1037] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:29:35 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:35.326470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9f857-7e69-448a-903b-ec6c05657bbd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:35.345905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:35.367312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2cf90-99d0-48c2-982f-a012e057926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:35.368091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2cf90-99d0-48c2-982f-a012e057926d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:35.368379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2cf90-99d0-48c2-982f-a012e057926d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:35.368656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2cf90-99d0-48c2-982f-a012e057926d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:35.377801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2cf90-99d0-48c2-982f-a012e057926d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:35.379285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2cf90-99d0-48c2-982f-a012e057926d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-22d2cf90-99d0-48c2-982f-a012e057926d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:35.397035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:35.398513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:35.399651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 515/1038] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 13 20:29:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:35.404319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fda0f-04eb-43e2-9c1b-67e98e5e1b20 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:35.513524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fda0f-04eb-43e2-9c1b-67e98e5e1b20 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:35.547520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b3239-cd53-46f6-8fd2-40b2d506f5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:35.548225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b3239-cd53-46f6-8fd2-40b2d506f5cd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:35.548523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b3239-cd53-46f6-8fd2-40b2d506f5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:35.548801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b3239-cd53-46f6-8fd2-40b2d506f5cd None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:35.549642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b3239-cd53-46f6-8fd2-40b2d506f5cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:35.549959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b3239-cd53-46f6-8fd2-40b2d506f5cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:35.551395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b3239-cd53-46f6-8fd2-40b2d506f5cd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:35.568948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 524/1039] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 13 20:29:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:35.575373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2cf90-99d0-48c2-982f-a012e057926d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:35.677981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2cf90-99d0-48c2-982f-a012e057926d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:35.713987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eeeda2-22ab-4571-9821-14db69d76109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:35.715041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeeda2-22ab-4571-9821-14db69d76109 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:35.715558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeeda2-22ab-4571-9821-14db69d76109 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:35.716037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeeda2-22ab-4571-9821-14db69d76109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:35.763172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:35.788625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eeeda2-22ab-4571-9821-14db69d76109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:35.790109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eeeda2-22ab-4571-9821-14db69d76109 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:35.892667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 516/1040] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:35 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:35.894386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b3239-cd53-46f6-8fd2-40b2d506f5cd None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:35.924052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:36.002480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b3239-cd53-46f6-8fd2-40b2d506f5cd None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:36.039632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724ff55-3f98-4497-a5d5-d848e7f58b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:36.040430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724ff55-3f98-4497-a5d5-d848e7f58b7a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.040794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724ff55-3f98-4497-a5d5-d848e7f58b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.041062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724ff55-3f98-4497-a5d5-d848e7f58b7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.055340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724ff55-3f98-4497-a5d5-d848e7f58b7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:36.056891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724ff55-3f98-4497-a5d5-d848e7f58b7a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-b724ff55-3f98-4497-a5d5-d848e7f58b7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:36.075673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:36.076547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:36.077470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:36.077470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:36.077470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:36.077470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:36.079555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 525/1041] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:36.082887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eeeda2-22ab-4571-9821-14db69d76109 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:36.160455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:36.185877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eeeda2-22ab-4571-9821-14db69d76109 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:36.221109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8381cb-0cbc-4ff8-8d25-a9454bf6a285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:36.221764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8381cb-0cbc-4ff8-8d25-a9454bf6a285 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.222047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8381cb-0cbc-4ff8-8d25-a9454bf6a285 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.222299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8381cb-0cbc-4ff8-8d25-a9454bf6a285 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.223039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8381cb-0cbc-4ff8-8d25-a9454bf6a285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:36.223341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8381cb-0cbc-4ff8-8d25-a9454bf6a285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:36.224762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8381cb-0cbc-4ff8-8d25-a9454bf6a285 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:36.241021 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 517/1042] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:36.242673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724ff55-3f98-4497-a5d5-d848e7f58b7a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:36.327010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:36.334379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e095f-7c4a-4ca6-8c4e-1e359f468d15 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.334916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e095f-7c4a-4ca6-8c4e-1e359f468d15 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.335200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e095f-7c4a-4ca6-8c4e-1e359f468d15 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.337385 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 518/1043] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:36.342879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac441b7f-0cf7-488e-a3fa-9f1f0828f6e4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.343202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac441b7f-0cf7-488e-a3fa-9f1f0828f6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.343438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac441b7f-0cf7-488e-a3fa-9f1f0828f6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.348475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724ff55-3f98-4497-a5d5-d848e7f58b7a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:36.359960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ac441b7f-0cf7-488e-a3fa-9f1f0828f6e4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:36.361138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ac441b7f-0cf7-488e-a3fa-9f1f0828f6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:36.387084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44a906-3f05-4938-a2eb-a178ca5b2b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:36.387913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44a906-3f05-4938-a2eb-a178ca5b2b35 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.388202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44a906-3f05-4938-a2eb-a178ca5b2b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.388463 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44a906-3f05-4938-a2eb-a178ca5b2b35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.395397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44a906-3f05-4938-a2eb-a178ca5b2b35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:36.396881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44a906-3f05-4938-a2eb-a178ca5b2b35 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3a44a906-3f05-4938-a2eb-a178ca5b2b35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:36.410823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:36.411995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:36.412857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:36.412857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:36.412857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:36.412857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:36.412857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:36.412857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:36.412857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:36.412857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:36.416746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 526/1044] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:36.417937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44a906-3f05-4938-a2eb-a178ca5b2b35 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:36.432947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 519/1045] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:36.456070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac441b7f-0cf7-488e-a3fa-9f1f0828f6e4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:36.525178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44a906-3f05-4938-a2eb-a178ca5b2b35 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:36.525707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:36.562228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6f9a0-c1cc-4b9c-8c7e-32519de5f8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:36.562491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac441b7f-0cf7-488e-a3fa-9f1f0828f6e4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:36.563362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6f9a0-c1cc-4b9c-8c7e-32519de5f8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.564024 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6f9a0-c1cc-4b9c-8c7e-32519de5f8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.564500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6f9a0-c1cc-4b9c-8c7e-32519de5f8d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.566168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6f9a0-c1cc-4b9c-8c7e-32519de5f8d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:36.566717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6f9a0-c1cc-4b9c-8c7e-32519de5f8d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:36.568380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6f9a0-c1cc-4b9c-8c7e-32519de5f8d7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:36.578702 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:36.588043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 527/1046] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:36.589120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6f9a0-c1cc-4b9c-8c7e-32519de5f8d7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:36.596864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4da236-d396-4102-ad01-f3dcb8f04fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:36.597659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4da236-d396-4102-ad01-f3dcb8f04fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.597940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4da236-d396-4102-ad01-f3dcb8f04fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.598218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4da236-d396-4102-ad01-f3dcb8f04fc8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.598863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4da236-d396-4102-ad01-f3dcb8f04fc8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:36.600270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4da236-d396-4102-ad01-f3dcb8f04fc8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1b4da236-d396-4102-ad01-f3dcb8f04fc8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:36.611647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:36.612633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:36.613491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 520/1047] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Jul 13 20:29:36 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:36.619143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4da236-d396-4102-ad01-f3dcb8f04fc8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:36.707683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6f9a0-c1cc-4b9c-8c7e-32519de5f8d7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:36.725797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4da236-d396-4102-ad01-f3dcb8f04fc8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:36.746501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a77caa-c084-461c-bb7b-c1e267655482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:36.747543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a77caa-c084-461c-bb7b-c1e267655482 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.748019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a77caa-c084-461c-bb7b-c1e267655482 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.748587 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a77caa-c084-461c-bb7b-c1e267655482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.749609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a77caa-c084-461c-bb7b-c1e267655482 None admin] RBAC: Adding query filter params (role.id=3c7426b3b0754e6089b5ccc02c8b3eec, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=e80bb5b10ce945cabbfb88ca5df92e3c) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:36.750118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a77caa-c084-461c-bb7b-c1e267655482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:36.751731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a77caa-c084-461c-bb7b-c1e267655482 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:36.755448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:36.762329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e25fa0-fd5b-4773-8b8f-a30b89a50a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:36.763008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e25fa0-fd5b-4773-8b8f-a30b89a50a2f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.763290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e25fa0-fd5b-4773-8b8f-a30b89a50a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.763724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e25fa0-fd5b-4773-8b8f-a30b89a50a2f None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.764321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e25fa0-fd5b-4773-8b8f-a30b89a50a2f None admin] RBAC: Adding query filter params (name=image) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:36.764645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e25fa0-fd5b-4773-8b8f-a30b89a50a2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:36.766328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e25fa0-fd5b-4773-8b8f-a30b89a50a2f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:36.767282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 528/1048] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:36 2022] GET /identity/v3/role_assignments?user.id=e80bb5b10ce945cabbfb88ca5df92e3c&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=3c7426b3b0754e6089b5ccc02c8b3eec => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:36.777539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 521/1049] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:36.778642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a77caa-c084-461c-bb7b-c1e267655482 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:36.854393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:36.898522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a77caa-c084-461c-bb7b-c1e267655482 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:36.935570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a41114-8dc7-4127-a6d4-d829e4dec2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:36.936197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a41114-8dc7-4127-a6d4-d829e4dec2ef None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:36.936485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a41114-8dc7-4127-a6d4-d829e4dec2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:36.936762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a41114-8dc7-4127-a6d4-d829e4dec2ef None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:36.937475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a41114-8dc7-4127-a6d4-d829e4dec2ef None admin] RBAC: Adding query filter params (type=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:36.937783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a41114-8dc7-4127-a6d4-d829e4dec2ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:36.939154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a41114-8dc7-4127-a6d4-d829e4dec2ef None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:36.947623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 529/1050] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:36.950495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e25fa0-fd5b-4773-8b8f-a30b89a50a2f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:37.056398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e25fa0-fd5b-4773-8b8f-a30b89a50a2f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:37.101522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bca9b-209b-48ac-91f0-435f654ae218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:37.102193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bca9b-209b-48ac-91f0-435f654ae218 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:37.102478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bca9b-209b-48ac-91f0-435f654ae218 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:37.102753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bca9b-209b-48ac-91f0-435f654ae218 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:37.103598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bca9b-209b-48ac-91f0-435f654ae218 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:37.105038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bca9b-209b-48ac-91f0-435f654ae218 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:37.127646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 522/1051] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 13 20:29:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:37.372467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9667943-fb13-4f2d-93db-423abff414e9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:37.372674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9667943-fb13-4f2d-93db-423abff414e9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:37.372940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9667943-fb13-4f2d-93db-423abff414e9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:37.374325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 530/1052] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:37.379876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae164a-9824-4800-ba00-16431d373804 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:37.380344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae164a-9824-4800-ba00-16431d373804 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:37.380631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae164a-9824-4800-ba00-16431d373804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:37.399074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-10ae164a-9824-4800-ba00-16431d373804 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:37.400427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-10ae164a-9824-4800-ba00-16431d373804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:37.454320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:37.484974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 523/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:37.508005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9667943-fb13-4f2d-93db-423abff414e9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:37.614692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9667943-fb13-4f2d-93db-423abff414e9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:37.641748 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:37.650263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a0727-9fb4-440e-8c87-1801dc6eb9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:37.651063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0727-9fb4-440e-8c87-1801dc6eb9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:37.651349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0727-9fb4-440e-8c87-1801dc6eb9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:37.651652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0727-9fb4-440e-8c87-1801dc6eb9b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:37.661950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a0727-9fb4-440e-8c87-1801dc6eb9b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-dd6a0727-9fb4-440e-8c87-1801dc6eb9b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:37.669682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:37.670597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:37.671431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:37.672865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 531/1054] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:29:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:37.676966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae164a-9824-4800-ba00-16431d373804 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:37.783344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae164a-9824-4800-ba00-16431d373804 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:37.817068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f0f91-dde9-41d6-b4f5-4f92a60a1d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:37.817740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f0f91-dde9-41d6-b4f5-4f92a60a1d2a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:37.818038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f0f91-dde9-41d6-b4f5-4f92a60a1d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:37.818294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f0f91-dde9-41d6-b4f5-4f92a60a1d2a None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:37.819032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f0f91-dde9-41d6-b4f5-4f92a60a1d2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:37.819336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f0f91-dde9-41d6-b4f5-4f92a60a1d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:37.820762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f0f91-dde9-41d6-b4f5-4f92a60a1d2a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:37.834931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 524/1055] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:29:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:37.836519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a0727-9fb4-440e-8c87-1801dc6eb9b6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:37.943314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a0727-9fb4-440e-8c87-1801dc6eb9b6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:37.977825 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999854b1-08ac-4fa2-a10d-fcfad7cf1970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:37.978836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999854b1-08ac-4fa2-a10d-fcfad7cf1970 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:37.979216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999854b1-08ac-4fa2-a10d-fcfad7cf1970 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:37.979624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999854b1-08ac-4fa2-a10d-fcfad7cf1970 None admin] PATH_INFO: `/v3/users/abb4f66894494988a7cc0a30d5d42792` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:37.992978 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999854b1-08ac-4fa2-a10d-fcfad7cf1970 None admin] RBAC: Authorizing `identity:get_user(user_id=abb4f66894494988a7cc0a30d5d42792)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:37.994579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999854b1-08ac-4fa2-a10d-fcfad7cf1970 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:37.996517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 532/1056] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:29:37 2022] GET /identity/v3/users/abb4f66894494988a7cc0a30d5d42792 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:38.002172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f0f91-dde9-41d6-b4f5-4f92a60a1d2a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:38.042472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:38.109855 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f0f91-dde9-41d6-b4f5-4f92a60a1d2a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:38.143227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d58cc-00b4-4c91-9576-978ff53bd463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:38.143931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d58cc-00b4-4c91-9576-978ff53bd463 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:38.144175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d58cc-00b4-4c91-9576-978ff53bd463 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:38.144482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d58cc-00b4-4c91-9576-978ff53bd463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:38.213353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d58cc-00b4-4c91-9576-978ff53bd463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:38.214815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d58cc-00b4-4c91-9576-978ff53bd463 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:38.254611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:38.320239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 525/1057] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:38.326446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999854b1-08ac-4fa2-a10d-fcfad7cf1970 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:38.374997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:38.429015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999854b1-08ac-4fa2-a10d-fcfad7cf1970 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:38.464088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ec678-0015-435c-921c-17099dab5905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:38.464880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ec678-0015-435c-921c-17099dab5905 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:38.465167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ec678-0015-435c-921c-17099dab5905 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:38.465447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ec678-0015-435c-921c-17099dab5905 None admin] PATH_INFO: `/v3/projects/e86088a77261404f8d193d34b4dc7ec1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:38.466688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ec678-0015-435c-921c-17099dab5905 None admin] RBAC: Authorizing `identity:get_project(project_id=e86088a77261404f8d193d34b4dc7ec1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:38.468191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ec678-0015-435c-921c-17099dab5905 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:38.470093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 533/1058] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:29:38 2022] GET /identity/v3/projects/e86088a77261404f8d193d34b4dc7ec1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:38.476034 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d58cc-00b4-4c91-9576-978ff53bd463 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:38.574434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d58cc-00b4-4c91-9576-978ff53bd463 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:38.608107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830963f6-b5e9-4768-b363-6e42b168a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:38.608763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830963f6-b5e9-4768-b363-6e42b168a911 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:38.609052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830963f6-b5e9-4768-b363-6e42b168a911 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:38.609329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830963f6-b5e9-4768-b363-6e42b168a911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:38.610165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830963f6-b5e9-4768-b363-6e42b168a911 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=e86088a77261404f8d193d34b4dc7ec1, user.id=abb4f66894494988a7cc0a30d5d42792) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:38.610481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830963f6-b5e9-4768-b363-6e42b168a911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:38.611956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830963f6-b5e9-4768-b363-6e42b168a911 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:38.617725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 526/1059] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:38 2022] GET /identity/v3/role_assignments?user.id=abb4f66894494988a7cc0a30d5d42792&scope.project.id=e86088a77261404f8d193d34b4dc7ec1&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:38.743111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c68bbe-1a63-492d-8aa5-a89c8a7422b7 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:38.743709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c68bbe-1a63-492d-8aa5-a89c8a7422b7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:38.743709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c68bbe-1a63-492d-8aa5-a89c8a7422b7 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:38.745493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 534/1060] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:38.752610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7378f-5656-48b8-bbea-557d68960c4a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:38.752963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7378f-5656-48b8-bbea-557d68960c4a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:38.753159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7378f-5656-48b8-bbea-557d68960c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:38.758344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:38.772708 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-19a7378f-5656-48b8-bbea-557d68960c4a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:38.773909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-19a7378f-5656-48b8-bbea-557d68960c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:38.847506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 527/1061] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:38.866800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c68bbe-1a63-492d-8aa5-a89c8a7422b7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:38.972718 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c68bbe-1a63-492d-8aa5-a89c8a7422b7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:38.987443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:39.008961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383532bb-1194-43fd-904f-0248e446052c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:39.009750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383532bb-1194-43fd-904f-0248e446052c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.010027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383532bb-1194-43fd-904f-0248e446052c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.010294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383532bb-1194-43fd-904f-0248e446052c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.020866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383532bb-1194-43fd-904f-0248e446052c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-383532bb-1194-43fd-904f-0248e446052c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:39.029181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:39.030076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:39.030908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:39.032317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 535/1062] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 13 20:29:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:39.036623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7378f-5656-48b8-bbea-557d68960c4a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:39.142884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087d80c-01b4-4b45-8930-f600cc49663d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.143235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087d80c-01b4-4b45-8930-f600cc49663d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.143778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087d80c-01b4-4b45-8930-f600cc49663d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.145989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 536/1063] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:39.148819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7378f-5656-48b8-bbea-557d68960c4a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:39.150506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aae7b6-1f17-4beb-a548-66e091bede99 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.150815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aae7b6-1f17-4beb-a548-66e091bede99 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.151093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aae7b6-1f17-4beb-a548-66e091bede99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.168038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-14aae7b6-1f17-4beb-a548-66e091bede99 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:39.169258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-14aae7b6-1f17-4beb-a548-66e091bede99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:39.183750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:39.189427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e1d93-b7d2-41d4-83ed-aa1e83e2ce59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:39.190105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e1d93-b7d2-41d4-83ed-aa1e83e2ce59 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.190391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e1d93-b7d2-41d4-83ed-aa1e83e2ce59 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.190668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e1d93-b7d2-41d4-83ed-aa1e83e2ce59 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.191857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e1d93-b7d2-41d4-83ed-aa1e83e2ce59 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:39.192170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e1d93-b7d2-41d4-83ed-aa1e83e2ce59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:39.193741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e1d93-b7d2-41d4-83ed-aa1e83e2ce59 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:39.207850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 528/1064] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 13 20:29:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:39.208735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e1d93-b7d2-41d4-83ed-aa1e83e2ce59 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:39.241988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 537/1065] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:39.263846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aae7b6-1f17-4beb-a548-66e091bede99 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:39.313597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e1d93-b7d2-41d4-83ed-aa1e83e2ce59 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:39.353273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc656a-7651-4468-bda5-745cc27fa361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:39.354864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc656a-7651-4468-bda5-745cc27fa361 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.355510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc656a-7651-4468-bda5-745cc27fa361 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.356051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc656a-7651-4468-bda5-745cc27fa361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.363164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aae7b6-1f17-4beb-a548-66e091bede99 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:39.366168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:39.368951 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc656a-7651-4468-bda5-745cc27fa361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:39.370635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc656a-7651-4468-bda5-745cc27fa361 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-46dc656a-7651-4468-bda5-745cc27fa361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:39.388913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:39.389802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:39.391269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:39.391269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:39.391269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:39.391269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:39.391269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 529/1066] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:39.395705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc656a-7651-4468-bda5-745cc27fa361 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:39.404938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b105c-9df9-4396-894c-c7597b7cc1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:39.405607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b105c-9df9-4396-894c-c7597b7cc1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.405891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b105c-9df9-4396-894c-c7597b7cc1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.406164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b105c-9df9-4396-894c-c7597b7cc1a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.442742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:39.477760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b105c-9df9-4396-894c-c7597b7cc1a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:39.479245 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b105c-9df9-4396-894c-c7597b7cc1a7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:39.497868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc656a-7651-4468-bda5-745cc27fa361 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:39.535102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf98183-5b55-46e8-8a72-18703f35771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:39.535778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf98183-5b55-46e8-8a72-18703f35771d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.536067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf98183-5b55-46e8-8a72-18703f35771d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.536349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf98183-5b55-46e8-8a72-18703f35771d None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.537066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf98183-5b55-46e8-8a72-18703f35771d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:39.537567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf98183-5b55-46e8-8a72-18703f35771d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:39.538755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf98183-5b55-46e8-8a72-18703f35771d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:39.555171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 530/1067] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:39.556719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf98183-5b55-46e8-8a72-18703f35771d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:39.586675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 538/1068] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:29:39.587876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b105c-9df9-4396-894c-c7597b7cc1a7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:39.591444 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:39.661768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf98183-5b55-46e8-8a72-18703f35771d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:39.670462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:39.695696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b105c-9df9-4396-894c-c7597b7cc1a7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:39.702126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cee3c-4695-4dcf-9ac1-29c878650a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:39.703429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cee3c-4695-4dcf-9ac1-29c878650a9a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.703656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cee3c-4695-4dcf-9ac1-29c878650a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.703931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cee3c-4695-4dcf-9ac1-29c878650a9a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.713949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cee3c-4695-4dcf-9ac1-29c878650a9a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:39.715411 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cee3c-4695-4dcf-9ac1-29c878650a9a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9c7cee3c-4695-4dcf-9ac1-29c878650a9a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:39.730559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:39.731526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:39.732469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 531/1069] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 13 20:29:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:39.737620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f100b5-61da-465d-91c4-7ce6d085463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:39.737759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cee3c-4695-4dcf-9ac1-29c878650a9a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:39.738417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f100b5-61da-465d-91c4-7ce6d085463c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.738712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f100b5-61da-465d-91c4-7ce6d085463c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.738988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f100b5-61da-465d-91c4-7ce6d085463c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.752455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f100b5-61da-465d-91c4-7ce6d085463c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:39.755353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f100b5-61da-465d-91c4-7ce6d085463c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-74f100b5-61da-465d-91c4-7ce6d085463c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:39.783076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:39.784156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:39.785793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:39.785793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:39.785793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:39.785793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:39.785793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 539/1070] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:39.791237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f100b5-61da-465d-91c4-7ce6d085463c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:39.845449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cee3c-4695-4dcf-9ac1-29c878650a9a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:39.859794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:39.881638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3f8db-d910-4345-b8cc-35cd11146a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:39.882314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3f8db-d910-4345-b8cc-35cd11146a6d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.882625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3f8db-d910-4345-b8cc-35cd11146a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.882889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3f8db-d910-4345-b8cc-35cd11146a6d None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.883766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3f8db-d910-4345-b8cc-35cd11146a6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:39.884083 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3f8db-d910-4345-b8cc-35cd11146a6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:39.885648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3f8db-d910-4345-b8cc-35cd11146a6d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:39.903155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 532/1071] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 13 20:29:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:39.908638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3f8db-d910-4345-b8cc-35cd11146a6d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:39.913003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f100b5-61da-465d-91c4-7ce6d085463c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:39.934770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:39.951207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6913b3f-ac61-47ef-ad6b-ee1d810312d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:39.951913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6913b3f-ac61-47ef-ad6b-ee1d810312d1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:39.952206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6913b3f-ac61-47ef-ad6b-ee1d810312d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:39.952489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6913b3f-ac61-47ef-ad6b-ee1d810312d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:39.953210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6913b3f-ac61-47ef-ad6b-ee1d810312d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:39.953522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6913b3f-ac61-47ef-ad6b-ee1d810312d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:39.954898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6913b3f-ac61-47ef-ad6b-ee1d810312d1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:39.975503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 540/1072] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:40.029318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3f8db-d910-4345-b8cc-35cd11146a6d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:40.077527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e744c73-6b79-4db6-8b28-7aedd6b1570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:40.078370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e744c73-6b79-4db6-8b28-7aedd6b1570b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:40.078659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e744c73-6b79-4db6-8b28-7aedd6b1570b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:40.078957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e744c73-6b79-4db6-8b28-7aedd6b1570b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:40.090692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e744c73-6b79-4db6-8b28-7aedd6b1570b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:40.092226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e744c73-6b79-4db6-8b28-7aedd6b1570b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0e744c73-6b79-4db6-8b28-7aedd6b1570b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:40.111187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:40.112237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:40.113128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:40.113128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:40.113128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:40.113128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:40.113128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 533/1073] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:40.118251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6913b3f-ac61-47ef-ad6b-ee1d810312d1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:40.238305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6913b3f-ac61-47ef-ad6b-ee1d810312d1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:40.255713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:40.273412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bc1dc-4067-4f4a-a1ed-a5338c8303c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:40.274083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bc1dc-4067-4f4a-a1ed-a5338c8303c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:40.274371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bc1dc-4067-4f4a-a1ed-a5338c8303c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:40.274651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bc1dc-4067-4f4a-a1ed-a5338c8303c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:40.275436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bc1dc-4067-4f4a-a1ed-a5338c8303c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:40.275781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bc1dc-4067-4f4a-a1ed-a5338c8303c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:40.277170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bc1dc-4067-4f4a-a1ed-a5338c8303c2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:40.288432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 541/1074] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:40.294440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e744c73-6b79-4db6-8b28-7aedd6b1570b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:40.373701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:40.396716 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e744c73-6b79-4db6-8b28-7aedd6b1570b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:40.429641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f8892-6801-4102-8883-dc1d038731d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:40.430444 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f8892-6801-4102-8883-dc1d038731d7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:40.430733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f8892-6801-4102-8883-dc1d038731d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:40.431013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f8892-6801-4102-8883-dc1d038731d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:40.437890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f8892-6801-4102-8883-dc1d038731d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:40.439336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f8892-6801-4102-8883-dc1d038731d7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c13f8892-6801-4102-8883-dc1d038731d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:40.453595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:40.454616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:40.455602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:40.455602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:40.455602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:40.455602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:40.455602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:40.455602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:40.455602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:40.455602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:40.455602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 534/1075] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:40.460775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bc1dc-4067-4f4a-a1ed-a5338c8303c2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:40.563356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bc1dc-4067-4f4a-a1ed-a5338c8303c2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:40.598047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310d45f-21f3-4768-a2dc-50bf84db518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:40.598769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310d45f-21f3-4768-a2dc-50bf84db518c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:40.599062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310d45f-21f3-4768-a2dc-50bf84db518c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:40.599340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310d45f-21f3-4768-a2dc-50bf84db518c None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:40.600160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310d45f-21f3-4768-a2dc-50bf84db518c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:40.600472 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310d45f-21f3-4768-a2dc-50bf84db518c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:40.601917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310d45f-21f3-4768-a2dc-50bf84db518c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:40.614186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 542/1076] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:40.620319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f8892-6801-4102-8883-dc1d038731d7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:40.625006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db098184-0e29-4023-95f5-9d371727dadb None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:40.625194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db098184-0e29-4023-95f5-9d371727dadb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:40.625522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db098184-0e29-4023-95f5-9d371727dadb None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:40.626834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 543/1077] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:40.631814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777e558-52d0-4619-bae6-2e55520747fa None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:40.632107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777e558-52d0-4619-bae6-2e55520747fa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:40.632395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777e558-52d0-4619-bae6-2e55520747fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:40.649559 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b777e558-52d0-4619-bae6-2e55520747fa None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:40.650791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b777e558-52d0-4619-bae6-2e55520747fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:40.681409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:40.719147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 544/1078] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:29:40.720844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f8892-6801-4102-8883-dc1d038731d7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:40.741025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777e558-52d0-4619-bae6-2e55520747fa None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:40.753477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08be32d-5a5e-4971-8d68-195186213924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:40.754381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08be32d-5a5e-4971-8d68-195186213924 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:40.754668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08be32d-5a5e-4971-8d68-195186213924 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:40.754947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08be32d-5a5e-4971-8d68-195186213924 None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/e80bb5b10ce945cabbfb88ca5df92e3c/roles/3c7426b3b0754e6089b5ccc02c8b3eec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:40.757277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08be32d-5a5e-4971-8d68-195186213924 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=e80bb5b10ce945cabbfb88ca5df92e3c, role_id=3c7426b3b0754e6089b5ccc02c8b3eec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:40.758735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08be32d-5a5e-4971-8d68-195186213924 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:40.761742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:40.773612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 535/1079] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:29:40 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/e80bb5b10ce945cabbfb88ca5df92e3c/roles/3c7426b3b0754e6089b5ccc02c8b3eec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:29:40.854066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777e558-52d0-4619-bae6-2e55520747fa None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:40.891485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7809ac-ce8c-4adc-a751-bd51cbb1a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:40.892222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7809ac-ce8c-4adc-a751-bd51cbb1a477 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:40.892508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7809ac-ce8c-4adc-a751-bd51cbb1a477 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:40.892784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7809ac-ce8c-4adc-a751-bd51cbb1a477 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:40.905206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7809ac-ce8c-4adc-a751-bd51cbb1a477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3b7809ac-ce8c-4adc-a751-bd51cbb1a477 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:40.912972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:40.914814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:40.915955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 545/1080] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:29:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:40.921771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08be32d-5a5e-4971-8d68-195186213924 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:41.020338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08be32d-5a5e-4971-8d68-195186213924 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:41.057839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7226ff0-d4d8-40c7-82f8-5cd6bf7a58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:41.058517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7226ff0-d4d8-40c7-82f8-5cd6bf7a58c9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:41.058798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7226ff0-d4d8-40c7-82f8-5cd6bf7a58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:41.059074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7226ff0-d4d8-40c7-82f8-5cd6bf7a58c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:41.059829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7226ff0-d4d8-40c7-82f8-5cd6bf7a58c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:41.060138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7226ff0-d4d8-40c7-82f8-5cd6bf7a58c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:41.061537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7226ff0-d4d8-40c7-82f8-5cd6bf7a58c9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:41.070582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 536/1081] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:29:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:41.076778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7809ac-ce8c-4adc-a751-bd51cbb1a477 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:41.106447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:41.180486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7809ac-ce8c-4adc-a751-bd51cbb1a477 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:41.214004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0d688-12e3-4191-958d-c6dcfa9d3eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:41.215023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0d688-12e3-4191-958d-c6dcfa9d3eea None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:41.215333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0d688-12e3-4191-958d-c6dcfa9d3eea None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:41.215642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0d688-12e3-4191-958d-c6dcfa9d3eea None admin] PATH_INFO: `/v3/users/abb4f66894494988a7cc0a30d5d42792` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:41.216947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0d688-12e3-4191-958d-c6dcfa9d3eea None admin] RBAC: Authorizing `identity:get_user(user_id=abb4f66894494988a7cc0a30d5d42792)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:41.218607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0d688-12e3-4191-958d-c6dcfa9d3eea None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:41.220447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 546/1082] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:29:41 2022] GET /identity/v3/users/abb4f66894494988a7cc0a30d5d42792 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:41.226873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7226ff0-d4d8-40c7-82f8-5cd6bf7a58c9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:41.289121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:41.332430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7226ff0-d4d8-40c7-82f8-5cd6bf7a58c9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:41.366558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4587e6-3aff-4d2c-9155-e4835ab5d423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:41.367420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4587e6-3aff-4d2c-9155-e4835ab5d423 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:41.367811 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4587e6-3aff-4d2c-9155-e4835ab5d423 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:41.368092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4587e6-3aff-4d2c-9155-e4835ab5d423 None admin] PATH_INFO: `/v3/projects/e86088a77261404f8d193d34b4dc7ec1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:41.369268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4587e6-3aff-4d2c-9155-e4835ab5d423 None admin] RBAC: Authorizing `identity:get_project(project_id=e86088a77261404f8d193d34b4dc7ec1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:41.370744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4587e6-3aff-4d2c-9155-e4835ab5d423 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:41.372467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 537/1083] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:29:41 2022] GET /identity/v3/projects/e86088a77261404f8d193d34b4dc7ec1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:41.378397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0d688-12e3-4191-958d-c6dcfa9d3eea None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:41.473851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:41.480983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0d688-12e3-4191-958d-c6dcfa9d3eea None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:41.516751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de2e36-01e3-4e2c-b04a-d3cadc172b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:41.517647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de2e36-01e3-4e2c-b04a-d3cadc172b56 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:41.517932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de2e36-01e3-4e2c-b04a-d3cadc172b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:41.518184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de2e36-01e3-4e2c-b04a-d3cadc172b56 None admin] PATH_INFO: `/v3/projects/e86088a77261404f8d193d34b4dc7ec1/users/abb4f66894494988a7cc0a30d5d42792/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:41.520223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de2e36-01e3-4e2c-b04a-d3cadc172b56 None admin] RBAC: Authorizing `identity:create_grant(project_id=e86088a77261404f8d193d34b4dc7ec1, user_id=abb4f66894494988a7cc0a30d5d42792, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:41.521674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de2e36-01e3-4e2c-b04a-d3cadc172b56 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:41.537251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 547/1084] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:29:41 2022] PUT /identity/v3/projects/e86088a77261404f8d193d34b4dc7ec1/users/abb4f66894494988a7cc0a30d5d42792/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:29:42.774026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6355639-02fe-4f6c-ad0c-5034bbadead7 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:42.774614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6355639-02fe-4f6c-ad0c-5034bbadead7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:42.774614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6355639-02fe-4f6c-ad0c-5034bbadead7 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:42.776037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 538/1085] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:42.781572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9686ea8-cbbe-4859-999f-d13030428aae None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:42.782003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9686ea8-cbbe-4859-999f-d13030428aae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:42.782114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9686ea8-cbbe-4859-999f-d13030428aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:42.799977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e9686ea8-cbbe-4859-999f-d13030428aae None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:42.801163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e9686ea8-cbbe-4859-999f-d13030428aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:42.868544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 548/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:42.892083 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6355639-02fe-4f6c-ad0c-5034bbadead7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:42.998092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6355639-02fe-4f6c-ad0c-5034bbadead7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:43.023573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:43.032578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1048d-2081-4de2-9115-c4a34eb1c787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:43.033316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1048d-2081-4de2-9115-c4a34eb1c787 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.033598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1048d-2081-4de2-9115-c4a34eb1c787 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.033868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1048d-2081-4de2-9115-c4a34eb1c787 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.043895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1048d-2081-4de2-9115-c4a34eb1c787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bdc1048d-2081-4de2-9115-c4a34eb1c787 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:43.052026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:43.052979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:43.053825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 539/1087] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 13 20:29:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:43.059290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9686ea8-cbbe-4859-999f-d13030428aae None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:43.147044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:43.164174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9686ea8-cbbe-4859-999f-d13030428aae None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:43.198292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2dfa60-fa2f-4689-9a4e-d189df073d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:43.198963 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dfa60-fa2f-4689-9a4e-d189df073d31 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.199249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dfa60-fa2f-4689-9a4e-d189df073d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.199551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dfa60-fa2f-4689-9a4e-d189df073d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.200300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2dfa60-fa2f-4689-9a4e-d189df073d31 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:43.200609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2dfa60-fa2f-4689-9a4e-d189df073d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:43.201984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2dfa60-fa2f-4689-9a4e-d189df073d31 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:43.210078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 549/1088] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 13 20:29:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:43.215892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1048d-2081-4de2-9115-c4a34eb1c787 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:43.313753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1048d-2081-4de2-9115-c4a34eb1c787 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:43.347960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0696c17-28cf-40fc-bbb9-94f909ba049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:43.348753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0696c17-28cf-40fc-bbb9-94f909ba049d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.349031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0696c17-28cf-40fc-bbb9-94f909ba049d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.349301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0696c17-28cf-40fc-bbb9-94f909ba049d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.360069 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0696c17-28cf-40fc-bbb9-94f909ba049d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:43.361523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0696c17-28cf-40fc-bbb9-94f909ba049d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b0696c17-28cf-40fc-bbb9-94f909ba049d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:43.378615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:43.379576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:43.380454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:43.380454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:43.380454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:43.380454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:43.380454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 540/1089] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:43.385952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2dfa60-fa2f-4689-9a4e-d189df073d31 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:43.486125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2dfa60-fa2f-4689-9a4e-d189df073d31 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:43.503484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b27046-95cf-4e17-80b1-8b5dc602336c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.503757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b27046-95cf-4e17-80b1-8b5dc602336c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.504065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b27046-95cf-4e17-80b1-8b5dc602336c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.505463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 541/1090] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:29:43.510438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1925-cd45-4dc8-ba3e-b2e81c82a90d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.510727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1925-cd45-4dc8-ba3e-b2e81c82a90d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.511004 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1925-cd45-4dc8-ba3e-b2e81c82a90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.519603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82803369-7a1a-4eeb-9f87-6fb7c08de19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:43.520255 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82803369-7a1a-4eeb-9f87-6fb7c08de19e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.520541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82803369-7a1a-4eeb-9f87-6fb7c08de19e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.520821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82803369-7a1a-4eeb-9f87-6fb7c08de19e None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.521537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82803369-7a1a-4eeb-9f87-6fb7c08de19e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:43.521846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82803369-7a1a-4eeb-9f87-6fb7c08de19e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:43.523218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82803369-7a1a-4eeb-9f87-6fb7c08de19e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:43.527423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-898c1925-cd45-4dc8-ba3e-b2e81c82a90d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:43.528647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-898c1925-cd45-4dc8-ba3e-b2e81c82a90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:43.534865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 550/1091] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:43.540531 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82803369-7a1a-4eeb-9f87-6fb7c08de19e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:43.549238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:43.559744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:43.597747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 542/1092] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:43.620926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c1925-cd45-4dc8-ba3e-b2e81c82a90d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:43.642905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82803369-7a1a-4eeb-9f87-6fb7c08de19e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:43.676244 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5fef5-1468-4fe6-9021-18659cc814e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:43.677049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5fef5-1468-4fe6-9021-18659cc814e1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.677328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5fef5-1468-4fe6-9021-18659cc814e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.677600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5fef5-1468-4fe6-9021-18659cc814e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.685077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5fef5-1468-4fe6-9021-18659cc814e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:43.686553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5fef5-1468-4fe6-9021-18659cc814e1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-54f5fef5-1468-4fe6-9021-18659cc814e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:29:43.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:43.702246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:43.703162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 551/1093] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 13 20:29:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:43.708237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5fef5-1468-4fe6-9021-18659cc814e1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:43.718849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c1925-cd45-4dc8-ba3e-b2e81c82a90d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:43.753935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd809d5-5f81-4f21-aacb-bf7557cb151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:43.754732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd809d5-5f81-4f21-aacb-bf7557cb151f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.754993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd809d5-5f81-4f21-aacb-bf7557cb151f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.755269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd809d5-5f81-4f21-aacb-bf7557cb151f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.765592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd809d5-5f81-4f21-aacb-bf7557cb151f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bdd809d5-5f81-4f21-aacb-bf7557cb151f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:29:43.776012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:43.777389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:29:43.778212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 543/1094] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:29:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:29:43.779750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:43.784713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd809d5-5f81-4f21-aacb-bf7557cb151f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:43.814492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:43.815114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5fef5-1468-4fe6-9021-18659cc814e1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:43.850891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab38a4-0f9a-45f0-b280-b70b66657d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:43.851591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab38a4-0f9a-45f0-b280-b70b66657d2a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.851794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab38a4-0f9a-45f0-b280-b70b66657d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.852069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab38a4-0f9a-45f0-b280-b70b66657d2a None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.853069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab38a4-0f9a-45f0-b280-b70b66657d2a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:43.853382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab38a4-0f9a-45f0-b280-b70b66657d2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:43.854839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab38a4-0f9a-45f0-b280-b70b66657d2a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:43.871953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 552/1095] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 13 20:29:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:43.877584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab38a4-0f9a-45f0-b280-b70b66657d2a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:43.884750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd809d5-5f81-4f21-aacb-bf7557cb151f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:43.916795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac2208-5656-4d17-90b4-d8f824c440c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:43.917459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac2208-5656-4d17-90b4-d8f824c440c3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:43.917741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac2208-5656-4d17-90b4-d8f824c440c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:43.917997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac2208-5656-4d17-90b4-d8f824c440c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:43.918752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac2208-5656-4d17-90b4-d8f824c440c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:43.919059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac2208-5656-4d17-90b4-d8f824c440c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:43.920466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac2208-5656-4d17-90b4-d8f824c440c3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:43.928298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 544/1096] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:29:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:43.934168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac2208-5656-4d17-90b4-d8f824c440c3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:43.976604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab38a4-0f9a-45f0-b280-b70b66657d2a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:43.999712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:44.010870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f8e47-3e3b-4f46-a5fe-17b7911884cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.011545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f8e47-3e3b-4f46-a5fe-17b7911884cc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.011826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f8e47-3e3b-4f46-a5fe-17b7911884cc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.012099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f8e47-3e3b-4f46-a5fe-17b7911884cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.022922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:44.032965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac2208-5656-4d17-90b4-d8f824c440c3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:44.071574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262eb00-ed8a-4ebd-a773-f29d4e286713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.072773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262eb00-ed8a-4ebd-a773-f29d4e286713 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.073054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262eb00-ed8a-4ebd-a773-f29d4e286713 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.073326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262eb00-ed8a-4ebd-a773-f29d4e286713 None admin] PATH_INFO: `/v3/users/abb4f66894494988a7cc0a30d5d42792` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.074503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262eb00-ed8a-4ebd-a773-f29d4e286713 None admin] RBAC: Authorizing `identity:get_user(user_id=abb4f66894494988a7cc0a30d5d42792)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.075984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262eb00-ed8a-4ebd-a773-f29d4e286713 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.077617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 545/1097] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:29:43 2022] GET /identity/v3/users/abb4f66894494988a7cc0a30d5d42792 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:44.078766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f8e47-3e3b-4f46-a5fe-17b7911884cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.080252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f8e47-3e3b-4f46-a5fe-17b7911884cc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.083213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262eb00-ed8a-4ebd-a773-f29d4e286713 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:44.178398 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 553/1098] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:44.178653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262eb00-ed8a-4ebd-a773-f29d4e286713 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:44.184075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f8e47-3e3b-4f46-a5fe-17b7911884cc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:44.207634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:44.213282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375b206-c4d4-44c7-be22-508d132bf193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.213927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375b206-c4d4-44c7-be22-508d132bf193 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.214218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375b206-c4d4-44c7-be22-508d132bf193 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.214490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375b206-c4d4-44c7-be22-508d132bf193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.241365 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:44.283712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375b206-c4d4-44c7-be22-508d132bf193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.285224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375b206-c4d4-44c7-be22-508d132bf193 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.285424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f8e47-3e3b-4f46-a5fe-17b7911884cc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:44.319497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65b8c3-3c73-4288-b597-0f42cb42fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.320316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65b8c3-3c73-4288-b597-0f42cb42fe34 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.320604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65b8c3-3c73-4288-b597-0f42cb42fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.320879 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65b8c3-3c73-4288-b597-0f42cb42fe34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.332397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65b8c3-3c73-4288-b597-0f42cb42fe34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.333833 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65b8c3-3c73-4288-b597-0f42cb42fe34 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3c65b8c3-3c73-4288-b597-0f42cb42fe34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:29:44.351751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:44.352949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:44.354077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:29:44.354077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:29:44.354077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:29:44.354077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:44.354077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 554/1099] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:44.358597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65b8c3-3c73-4288-b597-0f42cb42fe34 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:44.388830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 546/1100] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:29:44.394537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375b206-c4d4-44c7-be22-508d132bf193 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:44.435542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:44.451622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:44.458519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65b8c3-3c73-4288-b597-0f42cb42fe34 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:44.491350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839365-ec57-4c68-85a2-7412514c39fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.492044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839365-ec57-4c68-85a2-7412514c39fc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.492331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839365-ec57-4c68-85a2-7412514c39fc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.492439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375b206-c4d4-44c7-be22-508d132bf193 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:44.492611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839365-ec57-4c68-85a2-7412514c39fc None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.493331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839365-ec57-4c68-85a2-7412514c39fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:44.493673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839365-ec57-4c68-85a2-7412514c39fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.495055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839365-ec57-4c68-85a2-7412514c39fc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.506241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 555/1101] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:44.511884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839365-ec57-4c68-85a2-7412514c39fc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:44.527624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7ac57-3ccc-4c32-82d5-bf27dac6f90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.528484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7ac57-3ccc-4c32-82d5-bf27dac6f90e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.528765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7ac57-3ccc-4c32-82d5-bf27dac6f90e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.529067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7ac57-3ccc-4c32-82d5-bf27dac6f90e None admin] PATH_INFO: `/v3/projects/e86088a77261404f8d193d34b4dc7ec1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.530200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7ac57-3ccc-4c32-82d5-bf27dac6f90e None admin] RBAC: Authorizing `identity:get_project(project_id=e86088a77261404f8d193d34b4dc7ec1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.531653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7ac57-3ccc-4c32-82d5-bf27dac6f90e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.533337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 547/1102] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:29:44 2022] GET /identity/v3/projects/e86088a77261404f8d193d34b4dc7ec1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:44.539311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7ac57-3ccc-4c32-82d5-bf27dac6f90e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:44.609031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839365-ec57-4c68-85a2-7412514c39fc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:44.637118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7ac57-3ccc-4c32-82d5-bf27dac6f90e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:44.641410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5a014-1b77-4fbf-a762-1eb8c5075d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.642208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5a014-1b77-4fbf-a762-1eb8c5075d65 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.642491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5a014-1b77-4fbf-a762-1eb8c5075d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.642766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5a014-1b77-4fbf-a762-1eb8c5075d65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.645500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:44.650156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5a014-1b77-4fbf-a762-1eb8c5075d65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.651873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5a014-1b77-4fbf-a762-1eb8c5075d65 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.662822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-55b5a014-1b77-4fbf-a762-1eb8c5075d65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:29:44.667274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:29:44.668380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:29:44.669291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:29:44.669291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:29:44.669291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:29:44.669291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:29:44.669291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:29:44.669291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:29:44.669291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:29:44.669291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:29:44.669291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 556/1103] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:29:44.672063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55928e26-2db2-40c6-a8ba-091964e6b0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.672718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55928e26-2db2-40c6-a8ba-091964e6b0db None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.672998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55928e26-2db2-40c6-a8ba-091964e6b0db None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.673361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55928e26-2db2-40c6-a8ba-091964e6b0db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.674188 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55928e26-2db2-40c6-a8ba-091964e6b0db None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=e86088a77261404f8d193d34b4dc7ec1, user.id=abb4f66894494988a7cc0a30d5d42792) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:44.674308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5a014-1b77-4fbf-a762-1eb8c5075d65 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:44.674498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55928e26-2db2-40c6-a8ba-091964e6b0db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.675964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55928e26-2db2-40c6-a8ba-091964e6b0db None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.681966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 548/1104] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:44 2022] GET /identity/v3/role_assignments?user.id=abb4f66894494988a7cc0a30d5d42792&scope.project.id=e86088a77261404f8d193d34b4dc7ec1&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:44.784968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5a014-1b77-4fbf-a762-1eb8c5075d65 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:44.823750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93320f9-e7fc-43ec-8632-c9e6f7e97dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.824402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93320f9-e7fc-43ec-8632-c9e6f7e97dbc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.824675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93320f9-e7fc-43ec-8632-c9e6f7e97dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.824948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93320f9-e7fc-43ec-8632-c9e6f7e97dbc None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.825742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93320f9-e7fc-43ec-8632-c9e6f7e97dbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:44.826049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93320f9-e7fc-43ec-8632-c9e6f7e97dbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.827482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93320f9-e7fc-43ec-8632-c9e6f7e97dbc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.840044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 557/1105] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:29:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:29:44.846398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55928e26-2db2-40c6-a8ba-091964e6b0db None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:44.947974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55928e26-2db2-40c6-a8ba-091964e6b0db None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:44.982449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dba217-0d16-4a78-98d5-d5c703e08fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:44.983106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dba217-0d16-4a78-98d5-d5c703e08fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:44.983402 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dba217-0d16-4a78-98d5-d5c703e08fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:44.983717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dba217-0d16-4a78-98d5-d5c703e08fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:44.984546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dba217-0d16-4a78-98d5-d5c703e08fa1 None admin] RBAC: Adding query filter params (role.id=3c7426b3b0754e6089b5ccc02c8b3eec, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=e80bb5b10ce945cabbfb88ca5df92e3c) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:44.984825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dba217-0d16-4a78-98d5-d5c703e08fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:44.986297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dba217-0d16-4a78-98d5-d5c703e08fa1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:44.992469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 549/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:44 2022] GET /identity/v3/role_assignments?user.id=e80bb5b10ce945cabbfb88ca5df92e3c&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=3c7426b3b0754e6089b5ccc02c8b3eec => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:46.716993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba109d-fa60-4095-a439-d8b356648852 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:46.717828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba109d-fa60-4095-a439-d8b356648852 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:46.717975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba109d-fa60-4095-a439-d8b356648852 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:46.720194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 558/1107] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:46.725553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7de1a1-cb33-49a5-8a7f-1b2c46350865 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:46.725998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7de1a1-cb33-49a5-8a7f-1b2c46350865 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:46.726091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7de1a1-cb33-49a5-8a7f-1b2c46350865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:46.744740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cb7de1a1-cb33-49a5-8a7f-1b2c46350865 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:46.745911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cb7de1a1-cb33-49a5-8a7f-1b2c46350865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:46.806916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:46.826420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 550/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:46.850766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba109d-fa60-4095-a439-d8b356648852 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:46.908520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:46.955635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba109d-fa60-4095-a439-d8b356648852 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:46.991096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ee0f0-4f91-43f5-a4d0-a34dcad1eb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:46.991849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee0f0-4f91-43f5-a4d0-a34dcad1eb7a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:46.992141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee0f0-4f91-43f5-a4d0-a34dcad1eb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:46.992420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee0f0-4f91-43f5-a4d0-a34dcad1eb7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:46.993778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ee0f0-4f91-43f5-a4d0-a34dcad1eb7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:46.995220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ee0f0-4f91-43f5-a4d0-a34dcad1eb7a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:46.996841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 559/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:47.002876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7de1a1-cb33-49a5-8a7f-1b2c46350865 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:47.117087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7de1a1-cb33-49a5-8a7f-1b2c46350865 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:47.155063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30618a87-5f2a-45ca-9bab-83b13687fd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:47.155754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30618a87-5f2a-45ca-9bab-83b13687fd58 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:47.156037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30618a87-5f2a-45ca-9bab-83b13687fd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:47.156314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30618a87-5f2a-45ca-9bab-83b13687fd58 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:47.157024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30618a87-5f2a-45ca-9bab-83b13687fd58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:47.158464 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30618a87-5f2a-45ca-9bab-83b13687fd58 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:47.180140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 551/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:29:47 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:29:49.233657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91cbc-ba7c-4ff2-8b7c-884fcdaacbcb None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:49.234222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91cbc-ba7c-4ff2-8b7c-884fcdaacbcb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:49.234222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91cbc-ba7c-4ff2-8b7c-884fcdaacbcb None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:49.236358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 560/1111] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:49.245215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c388-402b-4893-beb6-78a6086a46a7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:49.245619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c388-402b-4893-beb6-78a6086a46a7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:49.245843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c388-402b-4893-beb6-78a6086a46a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:49.273064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1d01c388-402b-4893-beb6-78a6086a46a7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:49.274839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1d01c388-402b-4893-beb6-78a6086a46a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:49.314501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:49.352686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 552/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:49.376756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91cbc-ba7c-4ff2-8b7c-884fcdaacbcb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:49.483337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91cbc-ba7c-4ff2-8b7c-884fcdaacbcb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:49.512302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:49.525032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb301ceb-7a3c-44db-ab7f-b6f245defffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:49.525895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301ceb-7a3c-44db-ab7f-b6f245defffe None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:49.526179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301ceb-7a3c-44db-ab7f-b6f245defffe None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:49.526458 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301ceb-7a3c-44db-ab7f-b6f245defffe None admin] PATH_INFO: `/v3/roles/32bea78dbfe74b56bd5c1f113586c690` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:49.528041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb301ceb-7a3c-44db-ab7f-b6f245defffe None admin] RBAC: Authorizing `identity:get_role(role_id=32bea78dbfe74b56bd5c1f113586c690)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:49.529410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb301ceb-7a3c-44db-ab7f-b6f245defffe None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:49.530771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 561/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:29:49 2022] GET /identity/v3/roles/32bea78dbfe74b56bd5c1f113586c690 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:49.536846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c388-402b-4893-beb6-78a6086a46a7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:49.652940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c388-402b-4893-beb6-78a6086a46a7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:49.690259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311e4a-2de0-443e-b886-ea33754f776e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:49.691129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311e4a-2de0-443e-b886-ea33754f776e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:49.691416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311e4a-2de0-443e-b886-ea33754f776e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:49.691726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311e4a-2de0-443e-b886-ea33754f776e None admin] PATH_INFO: `/v3/users/014cdb3aae91436eb0b171dca686f084` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:49.705701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1311e4a-2de0-443e-b886-ea33754f776e None admin] RBAC: Authorizing `identity:get_user(user_id=014cdb3aae91436eb0b171dca686f084)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:49.707154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1311e4a-2de0-443e-b886-ea33754f776e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:49.708836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 553/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:29:49 2022] GET /identity/v3/users/014cdb3aae91436eb0b171dca686f084 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:49.715353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb301ceb-7a3c-44db-ab7f-b6f245defffe None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:49.838738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb301ceb-7a3c-44db-ab7f-b6f245defffe None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:49.874609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d3986-905c-4622-bcdd-d76549f16908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:49.875257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d3986-905c-4622-bcdd-d76549f16908 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:49.875565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d3986-905c-4622-bcdd-d76549f16908 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:49.875846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d3986-905c-4622-bcdd-d76549f16908 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:49.948217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:49.963481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d3986-905c-4622-bcdd-d76549f16908 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:49.964951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d3986-905c-4622-bcdd-d76549f16908 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:50.071324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 562/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:50.077828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311e4a-2de0-443e-b886-ea33754f776e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:50.107999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:50.181161 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311e4a-2de0-443e-b886-ea33754f776e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:50.216312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734ff59-a4f1-48e9-ba8b-841c7418942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:50.217168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734ff59-a4f1-48e9-ba8b-841c7418942a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:50.217457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734ff59-a4f1-48e9-ba8b-841c7418942a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:50.217735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734ff59-a4f1-48e9-ba8b-841c7418942a None admin] PATH_INFO: `/v3/projects/e86088a77261404f8d193d34b4dc7ec1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:50.219027 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734ff59-a4f1-48e9-ba8b-841c7418942a None admin] RBAC: Authorizing `identity:get_project(project_id=e86088a77261404f8d193d34b4dc7ec1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:50.220505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734ff59-a4f1-48e9-ba8b-841c7418942a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:50.222188 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 554/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:29:50 2022] GET /identity/v3/projects/e86088a77261404f8d193d34b4dc7ec1 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:50.228896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d3986-905c-4622-bcdd-d76549f16908 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:50.327892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:50.343567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d3986-905c-4622-bcdd-d76549f16908 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:50.383631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d751dc-0f5b-4eaa-932d-af1375df7495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:50.384360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d751dc-0f5b-4eaa-932d-af1375df7495 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:50.384634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d751dc-0f5b-4eaa-932d-af1375df7495 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:50.384909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d751dc-0f5b-4eaa-932d-af1375df7495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:50.385735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d751dc-0f5b-4eaa-932d-af1375df7495 None admin] RBAC: Adding query filter params (role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=e86088a77261404f8d193d34b4dc7ec1, user.id=014cdb3aae91436eb0b171dca686f084) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:50.386049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d751dc-0f5b-4eaa-932d-af1375df7495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:50.387505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d751dc-0f5b-4eaa-932d-af1375df7495 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:50.392982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 563/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:50 2022] GET /identity/v3/role_assignments?user.id=014cdb3aae91436eb0b171dca686f084&scope.project.id=e86088a77261404f8d193d34b4dc7ec1&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:52.355718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c73b-0e77-415a-b6cc-c8c0b032c6b0 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:52.356288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c73b-0e77-415a-b6cc-c8c0b032c6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:52.356288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c73b-0e77-415a-b6cc-c8c0b032c6b0 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:52.357588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 555/1118] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:52.364031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6b972-e5b6-4905-bebb-405eab557f2b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:52.364766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6b972-e5b6-4905-bebb-405eab557f2b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:52.364897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6b972-e5b6-4905-bebb-405eab557f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:52.384560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-08f6b972-e5b6-4905-bebb-405eab557f2b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:52.385754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-08f6b972-e5b6-4905-bebb-405eab557f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:52.458622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 564/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:52.482667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c73b-0e77-415a-b6cc-c8c0b032c6b0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:52.584997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:52.592881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c73b-0e77-415a-b6cc-c8c0b032c6b0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:52.627516 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652292b0-48bc-4d2c-a333-51b6a4bc5cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:52.628308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652292b0-48bc-4d2c-a333-51b6a4bc5cee None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:52.628584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652292b0-48bc-4d2c-a333-51b6a4bc5cee None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:52.628858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652292b0-48bc-4d2c-a333-51b6a4bc5cee None admin] PATH_INFO: `/v3/roles/32bea78dbfe74b56bd5c1f113586c690` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:52.630359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652292b0-48bc-4d2c-a333-51b6a4bc5cee None admin] RBAC: Authorizing `identity:get_role(role_id=32bea78dbfe74b56bd5c1f113586c690)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:52.631755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652292b0-48bc-4d2c-a333-51b6a4bc5cee None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:52.633114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 556/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:29:52 2022] GET /identity/v3/roles/32bea78dbfe74b56bd5c1f113586c690 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:52.642002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6b972-e5b6-4905-bebb-405eab557f2b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:52.740982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:52.757575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6b972-e5b6-4905-bebb-405eab557f2b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:52.803480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4831-2c2e-4c7d-9276-3c8225cd151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:52.804384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4831-2c2e-4c7d-9276-3c8225cd151c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:52.804664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4831-2c2e-4c7d-9276-3c8225cd151c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:52.804937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4831-2c2e-4c7d-9276-3c8225cd151c None admin] PATH_INFO: `/v3/users/014cdb3aae91436eb0b171dca686f084` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:52.806199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4831-2c2e-4c7d-9276-3c8225cd151c None admin] RBAC: Authorizing `identity:get_user(user_id=014cdb3aae91436eb0b171dca686f084)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:52.807669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4831-2c2e-4c7d-9276-3c8225cd151c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:52.809295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 565/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:29:52 2022] GET /identity/v3/users/014cdb3aae91436eb0b171dca686f084 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:52.815493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652292b0-48bc-4d2c-a333-51b6a4bc5cee None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:52.916999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652292b0-48bc-4d2c-a333-51b6a4bc5cee None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:52.952585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d7aa0-7ad4-447a-826a-00331b613129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:52.953436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d7aa0-7ad4-447a-826a-00331b613129 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:52.953715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d7aa0-7ad4-447a-826a-00331b613129 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:52.953981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d7aa0-7ad4-447a-826a-00331b613129 None admin] PATH_INFO: `/v3/projects/e86088a77261404f8d193d34b4dc7ec1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:52.955271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d7aa0-7ad4-447a-826a-00331b613129 None admin] RBAC: Authorizing `identity:get_project(project_id=e86088a77261404f8d193d34b4dc7ec1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:52.956745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d7aa0-7ad4-447a-826a-00331b613129 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:52.958433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 557/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:29:52 2022] GET /identity/v3/projects/e86088a77261404f8d193d34b4dc7ec1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:52.965118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4831-2c2e-4c7d-9276-3c8225cd151c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:53.080285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4831-2c2e-4c7d-9276-3c8225cd151c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:53.123602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815170c-fad0-4fd5-b855-f0f01ea4b111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:53.124497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815170c-fad0-4fd5-b855-f0f01ea4b111 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:53.124781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815170c-fad0-4fd5-b855-f0f01ea4b111 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:53.125055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815170c-fad0-4fd5-b855-f0f01ea4b111 None admin] PATH_INFO: `/v3/projects/e86088a77261404f8d193d34b4dc7ec1/users/014cdb3aae91436eb0b171dca686f084/roles/32bea78dbfe74b56bd5c1f113586c690` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:53.127283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815170c-fad0-4fd5-b855-f0f01ea4b111 None admin] RBAC: Authorizing `identity:create_grant(project_id=e86088a77261404f8d193d34b4dc7ec1, user_id=014cdb3aae91436eb0b171dca686f084, role_id=32bea78dbfe74b56bd5c1f113586c690)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:53.128767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815170c-fad0-4fd5-b855-f0f01ea4b111 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:53.145809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 566/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:29:52 2022] PUT /identity/v3/projects/e86088a77261404f8d193d34b4dc7ec1/users/014cdb3aae91436eb0b171dca686f084/roles/32bea78dbfe74b56bd5c1f113586c690 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:29:55.115797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78fea9-d11d-4aa9-9cfa-8ce9c8ce9f4b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:55.116293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78fea9-d11d-4aa9-9cfa-8ce9c8ce9f4b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:55.116293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78fea9-d11d-4aa9-9cfa-8ce9c8ce9f4b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:55.117754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 558/1124] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:55.124122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8e730-4fdb-4268-adec-0383614e5703 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:55.124870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8e730-4fdb-4268-adec-0383614e5703 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:55.125023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8e730-4fdb-4268-adec-0383614e5703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:55.152349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-32f8e730-4fdb-4268-adec-0383614e5703 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:55.153545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-32f8e730-4fdb-4268-adec-0383614e5703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:55.163076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:55.228204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 567/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:55.251924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78fea9-d11d-4aa9-9cfa-8ce9c8ce9f4b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:55.288530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:55.377497 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78fea9-d11d-4aa9-9cfa-8ce9c8ce9f4b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:55.411871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2ca25-e927-4152-b97b-e7caad6991f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:55.412710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2ca25-e927-4152-b97b-e7caad6991f8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:55.412992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2ca25-e927-4152-b97b-e7caad6991f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:55.413270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2ca25-e927-4152-b97b-e7caad6991f8 None admin] PATH_INFO: `/v3/roles/32bea78dbfe74b56bd5c1f113586c690` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:55.414866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2ca25-e927-4152-b97b-e7caad6991f8 None admin] RBAC: Authorizing `identity:get_role(role_id=32bea78dbfe74b56bd5c1f113586c690)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:55.416382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2ca25-e927-4152-b97b-e7caad6991f8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:55.417797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 559/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:29:55 2022] GET /identity/v3/roles/32bea78dbfe74b56bd5c1f113586c690 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:55.423927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8e730-4fdb-4268-adec-0383614e5703 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:55.531860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8e730-4fdb-4268-adec-0383614e5703 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:55.567914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04170a1-7714-4889-8ada-1734b6001be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:55.568780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04170a1-7714-4889-8ada-1734b6001be2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:55.569064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04170a1-7714-4889-8ada-1734b6001be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:55.569336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04170a1-7714-4889-8ada-1734b6001be2 None admin] PATH_INFO: `/v3/users/014cdb3aae91436eb0b171dca686f084` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:55.570594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04170a1-7714-4889-8ada-1734b6001be2 None admin] RBAC: Authorizing `identity:get_user(user_id=014cdb3aae91436eb0b171dca686f084)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:55.572476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04170a1-7714-4889-8ada-1734b6001be2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:55.574093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 568/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:29:55 2022] GET /identity/v3/users/014cdb3aae91436eb0b171dca686f084 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:55.583279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2ca25-e927-4152-b97b-e7caad6991f8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:55.689011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:55.692633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2ca25-e927-4152-b97b-e7caad6991f8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:55.730150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e40bf-8cf7-480d-bbe6-2cd3267aeb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:55.730791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e40bf-8cf7-480d-bbe6-2cd3267aeb00 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:55.731079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e40bf-8cf7-480d-bbe6-2cd3267aeb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:55.731357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e40bf-8cf7-480d-bbe6-2cd3267aeb00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:55.808345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e40bf-8cf7-480d-bbe6-2cd3267aeb00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:55.809811 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e40bf-8cf7-480d-bbe6-2cd3267aeb00 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:55.914282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:55.919849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 560/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:29:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:29:55.929481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04170a1-7714-4889-8ada-1734b6001be2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:55.945809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:56.036173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04170a1-7714-4889-8ada-1734b6001be2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:56.083951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b284d-56ef-41a6-9172-d35dc07ed55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:56.085326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b284d-56ef-41a6-9172-d35dc07ed55f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:56.085807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b284d-56ef-41a6-9172-d35dc07ed55f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:56.086276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b284d-56ef-41a6-9172-d35dc07ed55f None admin] PATH_INFO: `/v3/projects/e86088a77261404f8d193d34b4dc7ec1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:56.087944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b284d-56ef-41a6-9172-d35dc07ed55f None admin] RBAC: Authorizing `identity:get_project(project_id=e86088a77261404f8d193d34b4dc7ec1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:56.089404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b284d-56ef-41a6-9172-d35dc07ed55f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:56.091090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 569/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:29:55 2022] GET /identity/v3/projects/e86088a77261404f8d193d34b4dc7ec1 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:56.097462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e40bf-8cf7-480d-bbe6-2cd3267aeb00 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:56.206699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e40bf-8cf7-480d-bbe6-2cd3267aeb00 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:56.246590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8f356-69ff-4e32-a512-a8acf02b22f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:56.247251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8f356-69ff-4e32-a512-a8acf02b22f4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:56.247572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8f356-69ff-4e32-a512-a8acf02b22f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:56.247845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8f356-69ff-4e32-a512-a8acf02b22f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:56.248678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8f356-69ff-4e32-a512-a8acf02b22f4 None admin] RBAC: Adding query filter params (role.id=32bea78dbfe74b56bd5c1f113586c690, scope.project.id=e86088a77261404f8d193d34b4dc7ec1, user.id=014cdb3aae91436eb0b171dca686f084) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:29:56.248991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8f356-69ff-4e32-a512-a8acf02b22f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:56.250446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8f356-69ff-4e32-a512-a8acf02b22f4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:56.256474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 561/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:29:56 2022] GET /identity/v3/role_assignments?user.id=014cdb3aae91436eb0b171dca686f084&scope.project.id=e86088a77261404f8d193d34b4dc7ec1&role.id=32bea78dbfe74b56bd5c1f113586c690 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:58.281227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fec515-c381-46ce-af39-520dc0dcd366 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:58.282047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fec515-c381-46ce-af39-520dc0dcd366 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:58.282195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fec515-c381-46ce-af39-520dc0dcd366 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:58.284675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 570/1131] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:29:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:29:58.291268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b9db-73a5-4bd8-8fd6-bb5e26739ad5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:58.291980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b9db-73a5-4bd8-8fd6-bb5e26739ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:58.292191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b9db-73a5-4bd8-8fd6-bb5e26739ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:58.315135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ddc2b9db-73a5-4bd8-8fd6-bb5e26739ad5 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:29:58.316319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ddc2b9db-73a5-4bd8-8fd6-bb5e26739ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:29:58.380108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:58.389818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 562/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:29:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:29:58.417590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fec515-c381-46ce-af39-520dc0dcd366 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:58.510385 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:29:58.533916 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fec515-c381-46ce-af39-520dc0dcd366 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:58.570081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fdda0-f8ce-44d6-ab90-f6b572fd264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:58.570807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fdda0-f8ce-44d6-ab90-f6b572fd264d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:58.571148 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fdda0-f8ce-44d6-ab90-f6b572fd264d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:58.571428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fdda0-f8ce-44d6-ab90-f6b572fd264d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:58.573076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fdda0-f8ce-44d6-ab90-f6b572fd264d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:58.574532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fdda0-f8ce-44d6-ab90-f6b572fd264d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:58.576181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 571/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:29:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:29:58.586092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b9db-73a5-4bd8-8fd6-bb5e26739ad5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:29:58.691516 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b9db-73a5-4bd8-8fd6-bb5e26739ad5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:29:58.727583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47d9e6-d90d-4316-af76-d336fabdc03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:29:58.728237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47d9e6-d90d-4316-af76-d336fabdc03c None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:29:58.728525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47d9e6-d90d-4316-af76-d336fabdc03c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:29:58.728806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47d9e6-d90d-4316-af76-d336fabdc03c None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:29:58.729521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47d9e6-d90d-4316-af76-d336fabdc03c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:29:58.730971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47d9e6-d90d-4316-af76-d336fabdc03c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:29:58.758940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 563/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:29:58 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:30:00.764631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e8db6-135c-46ec-aac0-749d047a89ee None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:00.765184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e8db6-135c-46ec-aac0-749d047a89ee None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:00.765184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e8db6-135c-46ec-aac0-749d047a89ee None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:00.766547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 572/1135] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:00.773451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e521fd2-9f62-4a9a-b1d3-cda76753531e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:00.774065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e521fd2-9f62-4a9a-b1d3-cda76753531e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:00.774208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e521fd2-9f62-4a9a-b1d3-cda76753531e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:00.798303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0e521fd2-9f62-4a9a-b1d3-cda76753531e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:00.799522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0e521fd2-9f62-4a9a-b1d3-cda76753531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:00.845221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:00.871814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 564/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:00.896766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e8db6-135c-46ec-aac0-749d047a89ee None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:01.014171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e8db6-135c-46ec-aac0-749d047a89ee None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:01.056046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93011476-dae9-46ce-ac8c-09aaba8063f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:01.057094 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011476-dae9-46ce-ac8c-09aaba8063f5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:01.057481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011476-dae9-46ce-ac8c-09aaba8063f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:01.057870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011476-dae9-46ce-ac8c-09aaba8063f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:01.059514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93011476-dae9-46ce-ac8c-09aaba8063f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:01.060974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93011476-dae9-46ce-ac8c-09aaba8063f5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:01.062565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 573/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:01.069266 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e521fd2-9f62-4a9a-b1d3-cda76753531e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:01.173085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e521fd2-9f62-4a9a-b1d3-cda76753531e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:01.209103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ee02a-1787-48aa-b7b2-40b75ac29960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:01.209765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ee02a-1787-48aa-b7b2-40b75ac29960 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:01.210053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ee02a-1787-48aa-b7b2-40b75ac29960 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:01.210345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ee02a-1787-48aa-b7b2-40b75ac29960 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:01.211047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ee02a-1787-48aa-b7b2-40b75ac29960 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:01.212609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ee02a-1787-48aa-b7b2-40b75ac29960 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:01.234327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 565/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:30:01 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:30:03.270567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ebd97-9c22-43b0-82a4-6f7a530cb2c7 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:03.271449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ebd97-9c22-43b0-82a4-6f7a530cb2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:03.271749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ebd97-9c22-43b0-82a4-6f7a530cb2c7 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:03.273798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 574/1139] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:03.280076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fc15d-aff7-4110-b304-37826a0bdeda None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:03.280518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fc15d-aff7-4110-b304-37826a0bdeda None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:03.280660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fc15d-aff7-4110-b304-37826a0bdeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:03.302116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2d1fc15d-aff7-4110-b304-37826a0bdeda None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:03.303313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2d1fc15d-aff7-4110-b304-37826a0bdeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:03.332490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:03.378016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 566/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:03.402237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ebd97-9c22-43b0-82a4-6f7a530cb2c7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:03.421502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:03.510751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ebd97-9c22-43b0-82a4-6f7a530cb2c7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:03.546370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc271414-7f85-4a4f-b320-1808cf1daa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:03.547172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc271414-7f85-4a4f-b320-1808cf1daa42 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:03.547482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc271414-7f85-4a4f-b320-1808cf1daa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:03.547779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc271414-7f85-4a4f-b320-1808cf1daa42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:03.558931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc271414-7f85-4a4f-b320-1808cf1daa42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bc271414-7f85-4a4f-b320-1808cf1daa42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:30:03.566610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:30:03.567758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:30:03.573064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 575/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:30:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:30:03.574765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fc15d-aff7-4110-b304-37826a0bdeda None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:03.677603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fc15d-aff7-4110-b304-37826a0bdeda None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:03.716653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072d4d-6c76-481f-8814-4b59f38bcf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:03.717305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072d4d-6c76-481f-8814-4b59f38bcf3e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:03.717589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072d4d-6c76-481f-8814-4b59f38bcf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:03.717864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072d4d-6c76-481f-8814-4b59f38bcf3e None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:03.718603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072d4d-6c76-481f-8814-4b59f38bcf3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:03.718916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072d4d-6c76-481f-8814-4b59f38bcf3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:03.720325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072d4d-6c76-481f-8814-4b59f38bcf3e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:03.729609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 567/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:30:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:03.735500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc271414-7f85-4a4f-b320-1808cf1daa42 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:03.822765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:03.848973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc271414-7f85-4a4f-b320-1808cf1daa42 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:03.883784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32425f20-4ee5-42ed-8aa6-2e6fdfe77d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:03.884646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32425f20-4ee5-42ed-8aa6-2e6fdfe77d5c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:03.884929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32425f20-4ee5-42ed-8aa6-2e6fdfe77d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:03.885213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32425f20-4ee5-42ed-8aa6-2e6fdfe77d5c None admin] PATH_INFO: `/v3/users/92c0ce561a304e24b10ba1f09f5021fe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:03.898644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32425f20-4ee5-42ed-8aa6-2e6fdfe77d5c None admin] RBAC: Authorizing `identity:get_user(user_id=92c0ce561a304e24b10ba1f09f5021fe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:03.900166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32425f20-4ee5-42ed-8aa6-2e6fdfe77d5c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:03.901788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 576/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:30:03 2022] GET /identity/v3/users/92c0ce561a304e24b10ba1f09f5021fe => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:03.907745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072d4d-6c76-481f-8814-4b59f38bcf3e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:03.936198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:04.032838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072d4d-6c76-481f-8814-4b59f38bcf3e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:04.068431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1182be-94fa-4515-aa5e-8490f33c9084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:04.069080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1182be-94fa-4515-aa5e-8490f33c9084 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:04.069374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1182be-94fa-4515-aa5e-8490f33c9084 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:04.069647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1182be-94fa-4515-aa5e-8490f33c9084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:04.140698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1182be-94fa-4515-aa5e-8490f33c9084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:04.142148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1182be-94fa-4515-aa5e-8490f33c9084 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:04.166835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:04.249157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 568/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:30:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:30:04.258575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32425f20-4ee5-42ed-8aa6-2e6fdfe77d5c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:04.380656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32425f20-4ee5-42ed-8aa6-2e6fdfe77d5c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:04.415848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfc5f6-b56e-43c3-9223-eb7622b4f96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:04.416712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfc5f6-b56e-43c3-9223-eb7622b4f96d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:04.416990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfc5f6-b56e-43c3-9223-eb7622b4f96d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:04.417262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfc5f6-b56e-43c3-9223-eb7622b4f96d None admin] PATH_INFO: `/v3/projects/954efdff01014aeeba244bc67212ee16` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:04.418495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfc5f6-b56e-43c3-9223-eb7622b4f96d None admin] RBAC: Authorizing `identity:get_project(project_id=954efdff01014aeeba244bc67212ee16)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:04.419957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfc5f6-b56e-43c3-9223-eb7622b4f96d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:04.421644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 577/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:30:04 2022] GET /identity/v3/projects/954efdff01014aeeba244bc67212ee16 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:04.428181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1182be-94fa-4515-aa5e-8490f33c9084 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:04.533936 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1182be-94fa-4515-aa5e-8490f33c9084 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:04.560170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:04.573675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12584eeb-2ec8-471f-b8fc-3f7cf30f5613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:04.574334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12584eeb-2ec8-471f-b8fc-3f7cf30f5613 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:04.574616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12584eeb-2ec8-471f-b8fc-3f7cf30f5613 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:04.574890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12584eeb-2ec8-471f-b8fc-3f7cf30f5613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:04.575755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12584eeb-2ec8-471f-b8fc-3f7cf30f5613 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=954efdff01014aeeba244bc67212ee16, user.id=92c0ce561a304e24b10ba1f09f5021fe) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:04.576071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12584eeb-2ec8-471f-b8fc-3f7cf30f5613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:04.577519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12584eeb-2ec8-471f-b8fc-3f7cf30f5613 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:04.583443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 569/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:30:04 2022] GET /identity/v3/role_assignments?user.id=92c0ce561a304e24b10ba1f09f5021fe&scope.project.id=954efdff01014aeeba244bc67212ee16&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:06.574429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f3bc3-cd3e-46f7-88de-48914b01b8b1 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:06.575319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f3bc3-cd3e-46f7-88de-48914b01b8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:06.575319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f3bc3-cd3e-46f7-88de-48914b01b8b1 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:06.577551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 578/1147] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:06.582982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686da873-e5f6-42d3-840f-944fe00715c2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:06.583416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686da873-e5f6-42d3-840f-944fe00715c2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:06.583546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686da873-e5f6-42d3-840f-944fe00715c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:06.601785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-686da873-e5f6-42d3-840f-944fe00715c2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:06.602995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-686da873-e5f6-42d3-840f-944fe00715c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:06.678521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 570/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:06.702085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f3bc3-cd3e-46f7-88de-48914b01b8b1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:06.706712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:06.806153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f3bc3-cd3e-46f7-88de-48914b01b8b1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:06.841627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30caaeb9-4032-43aa-8058-9ca150046aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:06.842344 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30caaeb9-4032-43aa-8058-9ca150046aef None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:06.842620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30caaeb9-4032-43aa-8058-9ca150046aef None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:06.842895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30caaeb9-4032-43aa-8058-9ca150046aef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:06.853691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30caaeb9-4032-43aa-8058-9ca150046aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-30caaeb9-4032-43aa-8058-9ca150046aef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:30:06.861249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:30:06.862329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:30:06.863342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 579/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:30:06.868717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686da873-e5f6-42d3-840f-944fe00715c2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:06.970846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686da873-e5f6-42d3-840f-944fe00715c2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:06.988247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:07.006110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf7151-f456-400a-90c9-4c54fa91263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:07.006767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf7151-f456-400a-90c9-4c54fa91263a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:07.007052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf7151-f456-400a-90c9-4c54fa91263a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:07.007331 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf7151-f456-400a-90c9-4c54fa91263a None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:07.008185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf7151-f456-400a-90c9-4c54fa91263a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:07.008501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf7151-f456-400a-90c9-4c54fa91263a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:07.009888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf7151-f456-400a-90c9-4c54fa91263a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:07.018010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 571/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:07.023969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30caaeb9-4032-43aa-8058-9ca150046aef None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:07.088381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:07.130994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30caaeb9-4032-43aa-8058-9ca150046aef None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:07.168081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f4a6e-39f4-4178-b866-6f881f25ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:07.168952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f4a6e-39f4-4178-b866-6f881f25ce63 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:07.169235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f4a6e-39f4-4178-b866-6f881f25ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:07.169510 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f4a6e-39f4-4178-b866-6f881f25ce63 None admin] PATH_INFO: `/v3/users/92c0ce561a304e24b10ba1f09f5021fe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:07.170729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f4a6e-39f4-4178-b866-6f881f25ce63 None admin] RBAC: Authorizing `identity:get_user(user_id=92c0ce561a304e24b10ba1f09f5021fe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:07.172202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f4a6e-39f4-4178-b866-6f881f25ce63 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:07.173812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 580/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:30:07 2022] GET /identity/v3/users/92c0ce561a304e24b10ba1f09f5021fe => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:07.180212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf7151-f456-400a-90c9-4c54fa91263a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:07.296005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf7151-f456-400a-90c9-4c54fa91263a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:07.330916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c42ca-06de-4df8-8759-c8f7020a9323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:07.331805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c42ca-06de-4df8-8759-c8f7020a9323 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:07.332091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c42ca-06de-4df8-8759-c8f7020a9323 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:07.332366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c42ca-06de-4df8-8759-c8f7020a9323 None admin] PATH_INFO: `/v3/projects/954efdff01014aeeba244bc67212ee16` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:07.333659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c42ca-06de-4df8-8759-c8f7020a9323 None admin] RBAC: Authorizing `identity:get_project(project_id=954efdff01014aeeba244bc67212ee16)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:07.335105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c42ca-06de-4df8-8759-c8f7020a9323 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:07.336937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 572/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:30:07 2022] GET /identity/v3/projects/954efdff01014aeeba244bc67212ee16 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:30:07.345677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f4a6e-39f4-4178-b866-6f881f25ce63 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:07.452193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f4a6e-39f4-4178-b866-6f881f25ce63 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:07.481098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:07.487730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d92c8-0ac0-4865-a3f6-657e683c4412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:07.488614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d92c8-0ac0-4865-a3f6-657e683c4412 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:07.488894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d92c8-0ac0-4865-a3f6-657e683c4412 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:07.489172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d92c8-0ac0-4865-a3f6-657e683c4412 None admin] PATH_INFO: `/v3/projects/954efdff01014aeeba244bc67212ee16/users/92c0ce561a304e24b10ba1f09f5021fe/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:07.491125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d92c8-0ac0-4865-a3f6-657e683c4412 None admin] RBAC: Authorizing `identity:create_grant(project_id=954efdff01014aeeba244bc67212ee16, user_id=92c0ce561a304e24b10ba1f09f5021fe, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:07.492627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d92c8-0ac0-4865-a3f6-657e683c4412 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:07.510585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 581/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:30:07 2022] PUT /identity/v3/projects/954efdff01014aeeba244bc67212ee16/users/92c0ce561a304e24b10ba1f09f5021fe/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:30:09.474823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bebcd-a11e-4c33-919c-a2693f48f3bd None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:09.475353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bebcd-a11e-4c33-919c-a2693f48f3bd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:09.475488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bebcd-a11e-4c33-919c-a2693f48f3bd None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:09.477703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 573/1154] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:09.486269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6476c9-0e47-4257-8cfb-741520bbd1d8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:09.486674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6476c9-0e47-4257-8cfb-741520bbd1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:09.486900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6476c9-0e47-4257-8cfb-741520bbd1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:09.506501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cc6476c9-0e47-4257-8cfb-741520bbd1d8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:09.507727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cc6476c9-0e47-4257-8cfb-741520bbd1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:09.577206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 582/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:09.601087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bebcd-a11e-4c33-919c-a2693f48f3bd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:09.660760 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:09.717056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bebcd-a11e-4c33-919c-a2693f48f3bd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:09.752564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e65c7-0e02-4f27-814e-1d9a4b43bef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:09.753288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e65c7-0e02-4f27-814e-1d9a4b43bef5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:09.753577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e65c7-0e02-4f27-814e-1d9a4b43bef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:09.753857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e65c7-0e02-4f27-814e-1d9a4b43bef5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:09.765847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e65c7-0e02-4f27-814e-1d9a4b43bef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-812e65c7-0e02-4f27-814e-1d9a4b43bef5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:30:09.774450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:30:09.775659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:30:09.776724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 574/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:30:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:30:09.781900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6476c9-0e47-4257-8cfb-741520bbd1d8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:09.884271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6476c9-0e47-4257-8cfb-741520bbd1d8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:09.904287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:09.920153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2196c-d210-475b-a8c4-9e1eed532b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:09.920816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2196c-d210-475b-a8c4-9e1eed532b24 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:09.921102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2196c-d210-475b-a8c4-9e1eed532b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:09.921379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2196c-d210-475b-a8c4-9e1eed532b24 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:09.922125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2196c-d210-475b-a8c4-9e1eed532b24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:09.922438 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2196c-d210-475b-a8c4-9e1eed532b24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:09.923841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2196c-d210-475b-a8c4-9e1eed532b24 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:09.931823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 583/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:30:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:09.937609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e65c7-0e02-4f27-814e-1d9a4b43bef5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:10.074749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e65c7-0e02-4f27-814e-1d9a4b43bef5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:10.083788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:10.112245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf98b7e-9194-4693-a693-d758a98c621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:10.113119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf98b7e-9194-4693-a693-d758a98c621c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:10.113406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf98b7e-9194-4693-a693-d758a98c621c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:10.113678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf98b7e-9194-4693-a693-d758a98c621c None admin] PATH_INFO: `/v3/users/92c0ce561a304e24b10ba1f09f5021fe` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:10.114988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf98b7e-9194-4693-a693-d758a98c621c None admin] RBAC: Authorizing `identity:get_user(user_id=92c0ce561a304e24b10ba1f09f5021fe)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:10.116470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf98b7e-9194-4693-a693-d758a98c621c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:10.118335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 575/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:30:09 2022] GET /identity/v3/users/92c0ce561a304e24b10ba1f09f5021fe => generated 286 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:10.124564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2196c-d210-475b-a8c4-9e1eed532b24 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:10.229426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2196c-d210-475b-a8c4-9e1eed532b24 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:10.264385 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bbbc-f361-42ed-8675-25c2daac33b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:10.265020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bbbc-f361-42ed-8675-25c2daac33b7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:10.265302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bbbc-f361-42ed-8675-25c2daac33b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:10.265579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bbbc-f361-42ed-8675-25c2daac33b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:10.315296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:10.343188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9bbbc-f361-42ed-8675-25c2daac33b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:10.344792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9bbbc-f361-42ed-8675-25c2daac33b7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:10.456934 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 584/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:30:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:30:10.463836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf98b7e-9194-4693-a693-d758a98c621c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:10.569806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf98b7e-9194-4693-a693-d758a98c621c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:10.604456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65eef0-21b3-4680-a8a0-672419408d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:10.605301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65eef0-21b3-4680-a8a0-672419408d75 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:10.605580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65eef0-21b3-4680-a8a0-672419408d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:10.605851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65eef0-21b3-4680-a8a0-672419408d75 None admin] PATH_INFO: `/v3/projects/954efdff01014aeeba244bc67212ee16` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:10.607038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65eef0-21b3-4680-a8a0-672419408d75 None admin] RBAC: Authorizing `identity:get_project(project_id=954efdff01014aeeba244bc67212ee16)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:10.608503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65eef0-21b3-4680-a8a0-672419408d75 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:10.610188 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 576/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:30:10 2022] GET /identity/v3/projects/954efdff01014aeeba244bc67212ee16 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:10.616483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bbbc-f361-42ed-8675-25c2daac33b7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:10.704414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:10.731810 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bbbc-f361-42ed-8675-25c2daac33b7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:10.768043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c49a9-3681-45aa-9a53-6c6bb32849c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:10.768697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c49a9-3681-45aa-9a53-6c6bb32849c3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:10.768982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c49a9-3681-45aa-9a53-6c6bb32849c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:10.769261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c49a9-3681-45aa-9a53-6c6bb32849c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:10.770151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c49a9-3681-45aa-9a53-6c6bb32849c3 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=954efdff01014aeeba244bc67212ee16, user.id=92c0ce561a304e24b10ba1f09f5021fe) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:10.770465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c49a9-3681-45aa-9a53-6c6bb32849c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:10.771970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c49a9-3681-45aa-9a53-6c6bb32849c3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:10.777969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 585/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:30:10 2022] GET /identity/v3/role_assignments?user.id=92c0ce561a304e24b10ba1f09f5021fe&scope.project.id=954efdff01014aeeba244bc67212ee16&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:12.786229 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a4e5-4533-41a1-807c-b52a887c2e61 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:12.786748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a4e5-4533-41a1-807c-b52a887c2e61 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:12.786748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a4e5-4533-41a1-807c-b52a887c2e61 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:12.788423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 577/1162] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:12.793854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0c7ab-93fe-47de-8c9d-e8215d63b98a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:12.794285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0c7ab-93fe-47de-8c9d-e8215d63b98a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:12.794454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0c7ab-93fe-47de-8c9d-e8215d63b98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:12.812472 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-82c0c7ab-93fe-47de-8c9d-e8215d63b98a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:12.813669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-82c0c7ab-93fe-47de-8c9d-e8215d63b98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:12.880742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 586/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:12.903221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498a4e5-4533-41a1-807c-b52a887c2e61 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:12.955132 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:13.020172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498a4e5-4533-41a1-807c-b52a887c2e61 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:13.056130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848e88e-cd84-4359-8c6b-55f6eeb47e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:13.056713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848e88e-cd84-4359-8c6b-55f6eeb47e62 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:13.056999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848e88e-cd84-4359-8c6b-55f6eeb47e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:13.057275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848e88e-cd84-4359-8c6b-55f6eeb47e62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:13.130864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848e88e-cd84-4359-8c6b-55f6eeb47e62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:13.132354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848e88e-cd84-4359-8c6b-55f6eeb47e62 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:13.176445 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:13.232271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 578/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:30:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:30:13.239145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0c7ab-93fe-47de-8c9d-e8215d63b98a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:13.320027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:13.350570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0c7ab-93fe-47de-8c9d-e8215d63b98a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:13.395437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7e7c4-7452-43ef-8a83-76d56cd7c139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:13.396659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7e7c4-7452-43ef-8a83-76d56cd7c139 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:13.397048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7e7c4-7452-43ef-8a83-76d56cd7c139 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:13.397417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7e7c4-7452-43ef-8a83-76d56cd7c139 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:13.404845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7e7c4-7452-43ef-8a83-76d56cd7c139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:13.406298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7e7c4-7452-43ef-8a83-76d56cd7c139 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-52e7e7c4-7452-43ef-8a83-76d56cd7c139 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:30:13.420900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:30:13.422072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:30:13.422923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 587/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:30:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:30:13.428225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848e88e-cd84-4359-8c6b-55f6eeb47e62 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:13.535927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848e88e-cd84-4359-8c6b-55f6eeb47e62 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:13.576101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ff754-ef2c-4831-a698-92cd0e1d8be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:13.576694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ff754-ef2c-4831-a698-92cd0e1d8be7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:13.576980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ff754-ef2c-4831-a698-92cd0e1d8be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:13.577254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ff754-ef2c-4831-a698-92cd0e1d8be7 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:13.577971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ff754-ef2c-4831-a698-92cd0e1d8be7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:13.578276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ff754-ef2c-4831-a698-92cd0e1d8be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:13.579679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ff754-ef2c-4831-a698-92cd0e1d8be7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:13.586618 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 579/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 13 20:30:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:15.569648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66759f0a-fb59-4666-80bf-ecaef4a3a002 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:15.570212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66759f0a-fb59-4666-80bf-ecaef4a3a002 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:15.570212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66759f0a-fb59-4666-80bf-ecaef4a3a002 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:15.571998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 588/1167] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:15.578270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e1025-0ac9-4581-a228-8697290fb515 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:15.578943 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e1025-0ac9-4581-a228-8697290fb515 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:15.579161 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e1025-0ac9-4581-a228-8697290fb515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:15.600333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cb7e1025-0ac9-4581-a228-8697290fb515 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:15.601531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cb7e1025-0ac9-4581-a228-8697290fb515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:15.603500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:15.674511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 580/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:15.699724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66759f0a-fb59-4666-80bf-ecaef4a3a002 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:15.814524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66759f0a-fb59-4666-80bf-ecaef4a3a002 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:15.850170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cb366-d0e3-454b-99a6-e74a1c332379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:15.850762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cb366-d0e3-454b-99a6-e74a1c332379 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:15.851053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cb366-d0e3-454b-99a6-e74a1c332379 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:15.851330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cb366-d0e3-454b-99a6-e74a1c332379 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:15.852160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cb366-d0e3-454b-99a6-e74a1c332379 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:15.853539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cb366-d0e3-454b-99a6-e74a1c332379 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:15.857065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:15.876951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 589/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Wed Jul 13 20:30:15 2022] POST /identity/v3/domains => generated 267 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:30:17.914226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c069c03-2d38-4eba-8869-0680c8b5e411 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:17.914749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c069c03-2d38-4eba-8869-0680c8b5e411 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:17.914840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c069c03-2d38-4eba-8869-0680c8b5e411 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:17.916251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 581/1170] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:17.921815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d77836-dd28-4358-bb42-8163b3c7c9b2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:17.922257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d77836-dd28-4358-bb42-8163b3c7c9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:17.922364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d77836-dd28-4358-bb42-8163b3c7c9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:17.940111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b9d77836-dd28-4358-bb42-8163b3c7c9b2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:17.941305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b9d77836-dd28-4358-bb42-8163b3c7c9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:18.013554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 590/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:18.037795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c069c03-2d38-4eba-8869-0680c8b5e411 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:18.147903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c069c03-2d38-4eba-8869-0680c8b5e411 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:18.190921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa69d64-8480-4c70-bf89-974add3160f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:18.191751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa69d64-8480-4c70-bf89-974add3160f9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:18.192043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa69d64-8480-4c70-bf89-974add3160f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:18.192316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa69d64-8480-4c70-bf89-974add3160f9 None admin] PATH_INFO: `/v3/domains/45f77b81c64441afb43990373a4cf0ed` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:18.199675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:18.204432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa69d64-8480-4c70-bf89-974add3160f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=45f77b81c64441afb43990373a4cf0ed)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:18.205956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa69d64-8480-4c70-bf89-974add3160f9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:18.207554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 582/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Wed Jul 13 20:30:18 2022] GET /identity/v3/domains/45f77b81c64441afb43990373a4cf0ed => generated 267 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:18.220875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d77836-dd28-4358-bb42-8163b3c7c9b2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:18.324388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d77836-dd28-4358-bb42-8163b3c7c9b2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:18.350434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:18.359674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19f9c3-2486-4699-a97e-a6e17d7e0800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:18.360424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19f9c3-2486-4699-a97e-a6e17d7e0800 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:18.360713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19f9c3-2486-4699-a97e-a6e17d7e0800 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:18.360988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19f9c3-2486-4699-a97e-a6e17d7e0800 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:18.361708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19f9c3-2486-4699-a97e-a6e17d7e0800 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:18.363148 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19f9c3-2486-4699-a97e-a6e17d7e0800 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:18.396807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 591/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:30:18 2022] POST /identity/v3/projects => generated 361 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:30:20.407125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048fe9b-5260-4001-b47f-1ac59bd56c02 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:20.407932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048fe9b-5260-4001-b47f-1ac59bd56c02 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:20.407932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048fe9b-5260-4001-b47f-1ac59bd56c02 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:20.410055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 583/1174] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:20.417115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2580e-c62b-4d43-a7b0-af06edf354c9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:20.417550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2580e-c62b-4d43-a7b0-af06edf354c9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:20.417638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2580e-c62b-4d43-a7b0-af06edf354c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:20.434638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-51b2580e-c62b-4d43-a7b0-af06edf354c9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:20.435826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-51b2580e-c62b-4d43-a7b0-af06edf354c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:20.512558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 592/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:30:20.547492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048fe9b-5260-4001-b47f-1ac59bd56c02 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:20.654771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048fe9b-5260-4001-b47f-1ac59bd56c02 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:20.692593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cb077-f87e-46d3-9db5-db576116f04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:20.693810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cb077-f87e-46d3-9db5-db576116f04a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:20.694275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cb077-f87e-46d3-9db5-db576116f04a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:20.694697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cb077-f87e-46d3-9db5-db576116f04a None admin] PATH_INFO: `/v3/domains/45f77b81c64441afb43990373a4cf0ed` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:20.696944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08cb077-f87e-46d3-9db5-db576116f04a None admin] RBAC: Authorizing `identity:get_domain(domain_id=45f77b81c64441afb43990373a4cf0ed)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:20.699231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08cb077-f87e-46d3-9db5-db576116f04a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:20.701798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 584/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Wed Jul 13 20:30:20 2022] GET /identity/v3/domains/45f77b81c64441afb43990373a4cf0ed => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:20.707837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2580e-c62b-4d43-a7b0-af06edf354c9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:20.819275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2580e-c62b-4d43-a7b0-af06edf354c9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:20.828918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:20.854789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f5759-7ea9-4792-8447-2da6a87b22dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:20.855446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f5759-7ea9-4792-8447-2da6a87b22dc None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:20.855761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f5759-7ea9-4792-8447-2da6a87b22dc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:20.856041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f5759-7ea9-4792-8447-2da6a87b22dc None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:20.856751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f5759-7ea9-4792-8447-2da6a87b22dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:20.858189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f5759-7ea9-4792-8447-2da6a87b22dc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:20.879964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 593/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:30:20 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:30:22.922903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a3af4-9375-4b19-9cc7-9476a0fa85c4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:22.923576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a3af4-9375-4b19-9cc7-9476a0fa85c4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:22.923576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a3af4-9375-4b19-9cc7-9476a0fa85c4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:22.924843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 585/1178] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:22.932143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c2529-985f-4043-87b9-d03c340751d9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:22.932561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c2529-985f-4043-87b9-d03c340751d9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:22.932733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c2529-985f-4043-87b9-d03c340751d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:22.951205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-342c2529-985f-4043-87b9-d03c340751d9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:22.952433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-342c2529-985f-4043-87b9-d03c340751d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:23.021692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 594/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:23.044948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a3af4-9375-4b19-9cc7-9476a0fa85c4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:23.113255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:23.149435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a3af4-9375-4b19-9cc7-9476a0fa85c4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:23.181334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6307b33-1126-4372-b850-faabe4c7759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:23.182071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307b33-1126-4372-b850-faabe4c7759a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:23.182352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307b33-1126-4372-b850-faabe4c7759a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:23.182627 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307b33-1126-4372-b850-faabe4c7759a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:23.192598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6307b33-1126-4372-b850-faabe4c7759a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e6307b33-1126-4372-b850-faabe4c7759a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:30:23.200636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:30:23.201622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:30:23.202531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 586/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:30:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:30:23.208018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c2529-985f-4043-87b9-d03c340751d9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:23.298126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:23.305386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c2529-985f-4043-87b9-d03c340751d9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:23.341725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06234092-532f-4b55-ad5a-0b35487b7f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:23.342404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06234092-532f-4b55-ad5a-0b35487b7f3e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:23.342694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06234092-532f-4b55-ad5a-0b35487b7f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:23.342972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06234092-532f-4b55-ad5a-0b35487b7f3e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:23.343758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06234092-532f-4b55-ad5a-0b35487b7f3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:23.344071 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06234092-532f-4b55-ad5a-0b35487b7f3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:23.345457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06234092-532f-4b55-ad5a-0b35487b7f3e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:23.353720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 595/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:30:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:23.359592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6307b33-1126-4372-b850-faabe4c7759a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:23.460456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6307b33-1126-4372-b850-faabe4c7759a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:23.484961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:23.496700 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfb0bd-3e21-4d0d-b503-74ca944aa92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:23.497682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfb0bd-3e21-4d0d-b503-74ca944aa92e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:23.497968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfb0bd-3e21-4d0d-b503-74ca944aa92e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:23.498249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfb0bd-3e21-4d0d-b503-74ca944aa92e None admin] PATH_INFO: `/v3/users/ddfcab08187e43e4b4923bf57e426aae` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:23.511857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfb0bd-3e21-4d0d-b503-74ca944aa92e None admin] RBAC: Authorizing `identity:get_user(user_id=ddfcab08187e43e4b4923bf57e426aae)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:23.513325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfb0bd-3e21-4d0d-b503-74ca944aa92e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:23.515011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 587/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:30:23 2022] GET /identity/v3/users/ddfcab08187e43e4b4923bf57e426aae => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:30:23.520976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06234092-532f-4b55-ad5a-0b35487b7f3e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:23.623607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06234092-532f-4b55-ad5a-0b35487b7f3e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:23.657669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83bb7d-324f-49c1-8693-beafab026ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:23.658303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83bb7d-324f-49c1-8693-beafab026ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:23.658588 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83bb7d-324f-49c1-8693-beafab026ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:23.658863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83bb7d-324f-49c1-8693-beafab026ea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:23.683433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:23.728031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83bb7d-324f-49c1-8693-beafab026ea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:23.729418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83bb7d-324f-49c1-8693-beafab026ea2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:23.825913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 596/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:30:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:30:23.832712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfb0bd-3e21-4d0d-b503-74ca944aa92e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:23.940135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfb0bd-3e21-4d0d-b503-74ca944aa92e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:23.973348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24777f-d02a-4ac4-b389-e50b95bfe189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:23.974514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24777f-d02a-4ac4-b389-e50b95bfe189 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:23.974514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24777f-d02a-4ac4-b389-e50b95bfe189 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:23.974714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24777f-d02a-4ac4-b389-e50b95bfe189 None admin] PATH_INFO: `/v3/projects/623728c5fbe34bfd94b42f041d0425bc` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:23.975911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24777f-d02a-4ac4-b389-e50b95bfe189 None admin] RBAC: Authorizing `identity:get_project(project_id=623728c5fbe34bfd94b42f041d0425bc)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:23.977479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24777f-d02a-4ac4-b389-e50b95bfe189 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:23.979351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 588/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:30:23 2022] GET /identity/v3/projects/623728c5fbe34bfd94b42f041d0425bc => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:23.985723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83bb7d-324f-49c1-8693-beafab026ea2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:24.041903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:24.087981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83bb7d-324f-49c1-8693-beafab026ea2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:24.122035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090590-5c0f-4219-badd-4395ecb2c269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:24.122700 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090590-5c0f-4219-badd-4395ecb2c269 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:24.122987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090590-5c0f-4219-badd-4395ecb2c269 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:24.123266 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090590-5c0f-4219-badd-4395ecb2c269 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:24.124133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09090590-5c0f-4219-badd-4395ecb2c269 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=623728c5fbe34bfd94b42f041d0425bc, user.id=ddfcab08187e43e4b4923bf57e426aae) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:24.124446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09090590-5c0f-4219-badd-4395ecb2c269 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:24.125881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09090590-5c0f-4219-badd-4395ecb2c269 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:24.131374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 597/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:30:23 2022] GET /identity/v3/role_assignments?user.id=ddfcab08187e43e4b4923bf57e426aae&scope.project.id=623728c5fbe34bfd94b42f041d0425bc&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:30:26.115517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1fe46-bf6e-4bee-a329-d8ff30795508 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:26.116309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1fe46-bf6e-4bee-a329-d8ff30795508 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:26.116309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1fe46-bf6e-4bee-a329-d8ff30795508 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:26.117940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 589/1186] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:26.123290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25346a04-1428-4f4e-8d1a-82a7c37a2e69 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:26.123768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25346a04-1428-4f4e-8d1a-82a7c37a2e69 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:26.123867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25346a04-1428-4f4e-8d1a-82a7c37a2e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:26.142052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-25346a04-1428-4f4e-8d1a-82a7c37a2e69 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:26.143229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-25346a04-1428-4f4e-8d1a-82a7c37a2e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:26.215093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 598/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:26.238330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1fe46-bf6e-4bee-a329-d8ff30795508 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:26.299394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:26.352716 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1fe46-bf6e-4bee-a329-d8ff30795508 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:26.388219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11cfa8-5175-40b4-97d0-6dfb064e2402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:26.389009 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11cfa8-5175-40b4-97d0-6dfb064e2402 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:26.389298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11cfa8-5175-40b4-97d0-6dfb064e2402 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:26.389574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11cfa8-5175-40b4-97d0-6dfb064e2402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:26.400093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11cfa8-5175-40b4-97d0-6dfb064e2402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2a11cfa8-5175-40b4-97d0-6dfb064e2402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:30:26.407694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:30:26.408835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:30:26.409911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 590/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:30:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:30:26.415414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25346a04-1428-4f4e-8d1a-82a7c37a2e69 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:26.468994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:26.521902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25346a04-1428-4f4e-8d1a-82a7c37a2e69 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:26.558633 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95512a-c145-40cc-8100-1ff2e260ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:26.559303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95512a-c145-40cc-8100-1ff2e260ba7e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:26.559619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95512a-c145-40cc-8100-1ff2e260ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:26.559900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95512a-c145-40cc-8100-1ff2e260ba7e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:26.560650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95512a-c145-40cc-8100-1ff2e260ba7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:26.560959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95512a-c145-40cc-8100-1ff2e260ba7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:26.562337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95512a-c145-40cc-8100-1ff2e260ba7e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:26.570689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 599/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:30:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:26.577207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11cfa8-5175-40b4-97d0-6dfb064e2402 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:26.687975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11cfa8-5175-40b4-97d0-6dfb064e2402 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:26.701025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:26.723630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41f3ce-ca70-4eed-abd7-fdd2deb7031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:26.724494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41f3ce-ca70-4eed-abd7-fdd2deb7031b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:26.724778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41f3ce-ca70-4eed-abd7-fdd2deb7031b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:26.725062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41f3ce-ca70-4eed-abd7-fdd2deb7031b None admin] PATH_INFO: `/v3/users/ddfcab08187e43e4b4923bf57e426aae` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:26.726380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41f3ce-ca70-4eed-abd7-fdd2deb7031b None admin] RBAC: Authorizing `identity:get_user(user_id=ddfcab08187e43e4b4923bf57e426aae)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:26.727850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41f3ce-ca70-4eed-abd7-fdd2deb7031b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:26.729474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 591/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:30:26 2022] GET /identity/v3/users/ddfcab08187e43e4b4923bf57e426aae => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:26.737729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95512a-c145-40cc-8100-1ff2e260ba7e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:26.843248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95512a-c145-40cc-8100-1ff2e260ba7e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:26.856375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:26.883403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe341e26-ac87-4bb4-8cfc-3b58ccb1449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:26.884307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341e26-ac87-4bb4-8cfc-3b58ccb1449a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:26.884595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341e26-ac87-4bb4-8cfc-3b58ccb1449a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:26.884876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341e26-ac87-4bb4-8cfc-3b58ccb1449a None admin] PATH_INFO: `/v3/projects/623728c5fbe34bfd94b42f041d0425bc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:26.886155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe341e26-ac87-4bb4-8cfc-3b58ccb1449a None admin] RBAC: Authorizing `identity:get_project(project_id=623728c5fbe34bfd94b42f041d0425bc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:26.887634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe341e26-ac87-4bb4-8cfc-3b58ccb1449a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:26.889337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 600/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:30:26 2022] GET /identity/v3/projects/623728c5fbe34bfd94b42f041d0425bc => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:26.895487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41f3ce-ca70-4eed-abd7-fdd2deb7031b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:26.998372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41f3ce-ca70-4eed-abd7-fdd2deb7031b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:27.036981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb228da-bea5-428a-a7d2-ca963096b411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:27.037878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb228da-bea5-428a-a7d2-ca963096b411 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:27.038165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb228da-bea5-428a-a7d2-ca963096b411 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:27.038526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb228da-bea5-428a-a7d2-ca963096b411 None admin] PATH_INFO: `/v3/projects/623728c5fbe34bfd94b42f041d0425bc/users/ddfcab08187e43e4b4923bf57e426aae/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:27.040510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb228da-bea5-428a-a7d2-ca963096b411 None admin] RBAC: Authorizing `identity:create_grant(project_id=623728c5fbe34bfd94b42f041d0425bc, user_id=ddfcab08187e43e4b4923bf57e426aae, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:27.041973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb228da-bea5-428a-a7d2-ca963096b411 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:27.057635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 592/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:30:26 2022] PUT /identity/v3/projects/623728c5fbe34bfd94b42f041d0425bc/users/ddfcab08187e43e4b4923bf57e426aae/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:30:29.043611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ed4e-3343-4a5e-81e6-fc9afa6f802a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:29.044185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ed4e-3343-4a5e-81e6-fc9afa6f802a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:29.044185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ed4e-3343-4a5e-81e6-fc9afa6f802a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:29.045559 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 601/1193] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:30:29.050995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d9a3d-99f0-46e9-8a69-62c3880f6ad7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:29.051428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d9a3d-99f0-46e9-8a69-62c3880f6ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:29.051595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d9a3d-99f0-46e9-8a69-62c3880f6ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:29.069337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c27d9a3d-99f0-46e9-8a69-62c3880f6ad7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:30:29.070564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c27d9a3d-99f0-46e9-8a69-62c3880f6ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:30:29.106270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:29.143583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 593/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:30:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:30:29.168743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84ed4e-3343-4a5e-81e6-fc9afa6f802a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:29.288544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84ed4e-3343-4a5e-81e6-fc9afa6f802a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:29.324136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d201bb6-8782-49aa-8bf3-0e1d15ff1c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:29.324880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d201bb6-8782-49aa-8bf3-0e1d15ff1c7a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:29.325153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d201bb6-8782-49aa-8bf3-0e1d15ff1c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:29.325435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d201bb6-8782-49aa-8bf3-0e1d15ff1c7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:29.336448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d201bb6-8782-49aa-8bf3-0e1d15ff1c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8d201bb6-8782-49aa-8bf3-0e1d15ff1c7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:30:29.344171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:30:29.345201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:30:29.346417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 602/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:30:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:30:29.351942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d9a3d-99f0-46e9-8a69-62c3880f6ad7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:29.453533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d9a3d-99f0-46e9-8a69-62c3880f6ad7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:29.488264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5c41-e2ce-445a-b080-96697a9a48df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:29.488931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5c41-e2ce-445a-b080-96697a9a48df None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:29.489208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5c41-e2ce-445a-b080-96697a9a48df None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:29.489494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5c41-e2ce-445a-b080-96697a9a48df None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:29.490242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb5c41-e2ce-445a-b080-96697a9a48df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:29.490551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb5c41-e2ce-445a-b080-96697a9a48df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:29.491998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb5c41-e2ce-445a-b080-96697a9a48df None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:29.500245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 594/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:30:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:29.506330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d201bb6-8782-49aa-8bf3-0e1d15ff1c7a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:29.548682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:29.615817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d201bb6-8782-49aa-8bf3-0e1d15ff1c7a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:29.653058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f97474-8475-4422-8e6e-900244481948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:29.653935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f97474-8475-4422-8e6e-900244481948 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:29.654222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f97474-8475-4422-8e6e-900244481948 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:29.654503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f97474-8475-4422-8e6e-900244481948 None admin] PATH_INFO: `/v3/users/ddfcab08187e43e4b4923bf57e426aae` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:29.655840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f97474-8475-4422-8e6e-900244481948 None admin] RBAC: Authorizing `identity:get_user(user_id=ddfcab08187e43e4b4923bf57e426aae)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:29.657286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f97474-8475-4422-8e6e-900244481948 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:29.658931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 603/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 13 20:30:29 2022] GET /identity/v3/users/ddfcab08187e43e4b4923bf57e426aae => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:29.664815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5c41-e2ce-445a-b080-96697a9a48df None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:29.697729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:29.767128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5c41-e2ce-445a-b080-96697a9a48df None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:29.809363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ef461-9458-44be-a2ae-25918bf3890f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:29.810014 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ef461-9458-44be-a2ae-25918bf3890f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:29.810289 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ef461-9458-44be-a2ae-25918bf3890f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:29.810575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ef461-9458-44be-a2ae-25918bf3890f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:29.876126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ef461-9458-44be-a2ae-25918bf3890f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:29.877518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ef461-9458-44be-a2ae-25918bf3890f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:29.907661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:29.977244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 595/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:30:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:30:29.986494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f97474-8475-4422-8e6e-900244481948 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:30.102624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f97474-8475-4422-8e6e-900244481948 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:30.111652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:30.146695 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2824e0-8965-453e-82bd-b24dc86b5b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:30.147490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2824e0-8965-453e-82bd-b24dc86b5b1e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:30.147808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2824e0-8965-453e-82bd-b24dc86b5b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:30.148073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2824e0-8965-453e-82bd-b24dc86b5b1e None admin] PATH_INFO: `/v3/projects/623728c5fbe34bfd94b42f041d0425bc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:30.149319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2824e0-8965-453e-82bd-b24dc86b5b1e None admin] RBAC: Authorizing `identity:get_project(project_id=623728c5fbe34bfd94b42f041d0425bc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:30.150757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2824e0-8965-453e-82bd-b24dc86b5b1e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:30.152459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 604/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 13 20:30:29 2022] GET /identity/v3/projects/623728c5fbe34bfd94b42f041d0425bc => generated 361 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:30:30.159629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ef461-9458-44be-a2ae-25918bf3890f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:30:30.276552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ef461-9458-44be-a2ae-25918bf3890f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:30:30.317487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:30.324031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2d07f-79a2-4236-891a-c2321b9884ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:30:30.324695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2d07f-79a2-4236-891a-c2321b9884ef None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:30:30.324982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2d07f-79a2-4236-891a-c2321b9884ef None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:30:30.325265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2d07f-79a2-4236-891a-c2321b9884ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:30:30.326099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2d07f-79a2-4236-891a-c2321b9884ef None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=623728c5fbe34bfd94b42f041d0425bc, user.id=ddfcab08187e43e4b4923bf57e426aae) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:30:30.326416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2d07f-79a2-4236-891a-c2321b9884ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:30:30.327894 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2d07f-79a2-4236-891a-c2321b9884ef None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:30:30.333941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 596/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:30:30 2022] GET /identity/v3/role_assignments?user.id=ddfcab08187e43e4b4923bf57e426aae&scope.project.id=623728c5fbe34bfd94b42f041d0425bc&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:30:40.141316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=12 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:30:40.331150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:25.573377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71d9a-c5e4-436a-88c4-dedffadd472b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:25.573966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71d9a-c5e4-436a-88c4-dedffadd472b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:25.573966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71d9a-c5e4-436a-88c4-dedffadd472b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:25.575514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 605/1201] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:31:25.581051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c40b-4fec-429d-824c-fc941bca716e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:25.581505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c40b-4fec-429d-824c-fc941bca716e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:25.581821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c40b-4fec-429d-824c-fc941bca716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:25.599756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2ae0c40b-4fec-429d-824c-fc941bca716e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:31:25.600955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2ae0c40b-4fec-429d-824c-fc941bca716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:31:25.682890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 597/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:31:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:31:25.707393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de71d9a-c5e4-436a-88c4-dedffadd472b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:25.835489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de71d9a-c5e4-436a-88c4-dedffadd472b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:25.882884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7558e-f169-40f0-b58b-40fb263b07c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:25.883647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7558e-f169-40f0-b58b-40fb263b07c7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:25.883937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7558e-f169-40f0-b58b-40fb263b07c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:25.884238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7558e-f169-40f0-b58b-40fb263b07c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:25.897145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7558e-f169-40f0-b58b-40fb263b07c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:25.898976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7558e-f169-40f0-b58b-40fb263b07c7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9de7558e-f169-40f0-b58b-40fb263b07c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:25.920609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:25.921864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:25.927954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:25.927954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:25.927954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:25.927954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:25.927954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 606/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:25.929478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0c40b-4fec-429d-824c-fc941bca716e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:26.055817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0c40b-4fec-429d-824c-fc941bca716e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:26.090954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:26.098186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a665c9-b8a4-4f0d-8912-ae7d32e07fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:26.098963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a665c9-b8a4-4f0d-8912-ae7d32e07fbe None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:26.099246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a665c9-b8a4-4f0d-8912-ae7d32e07fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:26.099557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a665c9-b8a4-4f0d-8912-ae7d32e07fbe None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:26.100294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a665c9-b8a4-4f0d-8912-ae7d32e07fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:26.100654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a665c9-b8a4-4f0d-8912-ae7d32e07fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:26.102110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a665c9-b8a4-4f0d-8912-ae7d32e07fbe None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:26.119306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 598/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:26.124570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7558e-f169-40f0-b58b-40fb263b07c7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:26.202780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:26.268088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7558e-f169-40f0-b58b-40fb263b07c7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:26.312010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca001c-3846-43c5-a58f-3084414f3ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:26.312844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca001c-3846-43c5-a58f-3084414f3ac6 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:26.313132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca001c-3846-43c5-a58f-3084414f3ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:26.313443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca001c-3846-43c5-a58f-3084414f3ac6 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:26.314328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca001c-3846-43c5-a58f-3084414f3ac6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:26.316864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca001c-3846-43c5-a58f-3084414f3ac6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:26.351796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 607/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 13 20:31:26 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:31:28.571637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3178bf9-e6c7-4faf-b982-16c4f463423a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:28.572202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3178bf9-e6c7-4faf-b982-16c4f463423a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:28.572202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3178bf9-e6c7-4faf-b982-16c4f463423a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:28.573975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 599/1206] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:31:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:31:28.580390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355efb2-fcd5-48d6-abaa-f69bec79bbce None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:28.580886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355efb2-fcd5-48d6-abaa-f69bec79bbce None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:28.580981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355efb2-fcd5-48d6-abaa-f69bec79bbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:28.600829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4355efb2-fcd5-48d6-abaa-f69bec79bbce None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:31:28.602375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4355efb2-fcd5-48d6-abaa-f69bec79bbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:31:28.692899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 608/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:31:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:31:28.719291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3178bf9-e6c7-4faf-b982-16c4f463423a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:28.845512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3178bf9-e6c7-4faf-b982-16c4f463423a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:28.897776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674fa2c-35cf-4c5e-a2d0-70fae48c9372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:28.898676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674fa2c-35cf-4c5e-a2d0-70fae48c9372 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:28.899001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674fa2c-35cf-4c5e-a2d0-70fae48c9372 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:28.899286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674fa2c-35cf-4c5e-a2d0-70fae48c9372 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:28.911294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674fa2c-35cf-4c5e-a2d0-70fae48c9372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3674fa2c-35cf-4c5e-a2d0-70fae48c9372 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:31:28.920568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:28.921570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:31:28.922408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:28.923597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 600/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:31:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:31:28.930587 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355efb2-fcd5-48d6-abaa-f69bec79bbce None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:28.978891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:29.055731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355efb2-fcd5-48d6-abaa-f69bec79bbce None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:29.096307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f25c4c-aacc-4902-b410-9ac0631f6c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:29.096910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f25c4c-aacc-4902-b410-9ac0631f6c6b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:29.097199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f25c4c-aacc-4902-b410-9ac0631f6c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:29.097481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f25c4c-aacc-4902-b410-9ac0631f6c6b None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:29.098342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f25c4c-aacc-4902-b410-9ac0631f6c6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:29.098714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f25c4c-aacc-4902-b410-9ac0631f6c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:29.100227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f25c4c-aacc-4902-b410-9ac0631f6c6b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:29.108963 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 609/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:31:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:29.115799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674fa2c-35cf-4c5e-a2d0-70fae48c9372 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:29.178704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:29.237938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674fa2c-35cf-4c5e-a2d0-70fae48c9372 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:29.282349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978abc6-d4b7-4b7f-8a15-5be63d01fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:29.283351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978abc6-d4b7-4b7f-8a15-5be63d01fb85 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:29.283702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978abc6-d4b7-4b7f-8a15-5be63d01fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:29.284080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978abc6-d4b7-4b7f-8a15-5be63d01fb85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:29.297215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978abc6-d4b7-4b7f-8a15-5be63d01fb85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:29.299015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978abc6-d4b7-4b7f-8a15-5be63d01fb85 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7978abc6-d4b7-4b7f-8a15-5be63d01fb85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:29.320523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:29.321543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:29.322771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:29.322771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:29.322771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:29.322771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:29.322771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 601/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:31:29.328238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f25c4c-aacc-4902-b410-9ac0631f6c6b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:29.446277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:29.449619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f25c4c-aacc-4902-b410-9ac0631f6c6b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:29.494531 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b54f56-48dc-4b24-8e3b-5d5bff0298ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:29.495333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b54f56-48dc-4b24-8e3b-5d5bff0298ce None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:29.495663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b54f56-48dc-4b24-8e3b-5d5bff0298ce None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:29.495956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b54f56-48dc-4b24-8e3b-5d5bff0298ce None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:29.496681 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b54f56-48dc-4b24-8e3b-5d5bff0298ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:29.497008 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b54f56-48dc-4b24-8e3b-5d5bff0298ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:29.498471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b54f56-48dc-4b24-8e3b-5d5bff0298ce None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:29.511922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 610/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:31:29.517929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978abc6-d4b7-4b7f-8a15-5be63d01fb85 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:29.627631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978abc6-d4b7-4b7f-8a15-5be63d01fb85 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:29.629984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:29.666247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77616e-5b92-49f5-8ce1-d2d67abbf81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:29.667067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77616e-5b92-49f5-8ce1-d2d67abbf81a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:29.667657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77616e-5b92-49f5-8ce1-d2d67abbf81a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:29.667978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77616e-5b92-49f5-8ce1-d2d67abbf81a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:29.675549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77616e-5b92-49f5-8ce1-d2d67abbf81a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:29.677288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77616e-5b92-49f5-8ce1-d2d67abbf81a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ce77616e-5b92-49f5-8ce1-d2d67abbf81a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:31:29.692866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:29.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:29.696848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 602/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:29.700631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b54f56-48dc-4b24-8e3b-5d5bff0298ce None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:29.826867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b54f56-48dc-4b24-8e3b-5d5bff0298ce None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:29.867400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657278f2-a542-4bd2-9846-448e0a2cfa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:29.868428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657278f2-a542-4bd2-9846-448e0a2cfa3f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:29.868772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657278f2-a542-4bd2-9846-448e0a2cfa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:29.869216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657278f2-a542-4bd2-9846-448e0a2cfa3f None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:29.870169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657278f2-a542-4bd2-9846-448e0a2cfa3f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:29.870545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657278f2-a542-4bd2-9846-448e0a2cfa3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:29.872339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657278f2-a542-4bd2-9846-448e0a2cfa3f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:29.895229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 611/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 13 20:31:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:29.901511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77616e-5b92-49f5-8ce1-d2d67abbf81a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:30.019387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77616e-5b92-49f5-8ce1-d2d67abbf81a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:30.057643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbafee-fc02-4858-a6c2-306f0ed1fb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:30.058347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbafee-fc02-4858-a6c2-306f0ed1fb92 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:30.058637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbafee-fc02-4858-a6c2-306f0ed1fb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:30.058913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbafee-fc02-4858-a6c2-306f0ed1fb92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:30.093283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:30.138571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbafee-fc02-4858-a6c2-306f0ed1fb92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:30.140179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbafee-fc02-4858-a6c2-306f0ed1fb92 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:30.249347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 603/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:31:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:31:30.255353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657278f2-a542-4bd2-9846-448e0a2cfa3f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:30.275076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:30.360336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657278f2-a542-4bd2-9846-448e0a2cfa3f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:30.398467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d508d-e79f-4197-b924-6482ef25a2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:30.399292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d508d-e79f-4197-b924-6482ef25a2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:30.399578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d508d-e79f-4197-b924-6482ef25a2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:30.399860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d508d-e79f-4197-b924-6482ef25a2f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:30.411619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d508d-e79f-4197-b924-6482ef25a2f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:30.413084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d508d-e79f-4197-b924-6482ef25a2f8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-eb3d508d-e79f-4197-b924-6482ef25a2f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:30.431270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:30.432449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:30.433524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:30.433524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:30.433524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:30.433524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:30.433524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 612/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:30.438160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbafee-fc02-4858-a6c2-306f0ed1fb92 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:30.506656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:30.558455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbafee-fc02-4858-a6c2-306f0ed1fb92 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:30.595608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46672338-e1cd-495a-a519-5a34b6397f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:30.596485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46672338-e1cd-495a-a519-5a34b6397f4e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:30.596814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46672338-e1cd-495a-a519-5a34b6397f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:30.598208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46672338-e1cd-495a-a519-5a34b6397f4e None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:30.598208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46672338-e1cd-495a-a519-5a34b6397f4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:30.598208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46672338-e1cd-495a-a519-5a34b6397f4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:30.599836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46672338-e1cd-495a-a519-5a34b6397f4e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:30.613315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 604/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:31:30.619203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d508d-e79f-4197-b924-6482ef25a2f8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:30.702712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:30.737347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d508d-e79f-4197-b924-6482ef25a2f8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:30.774467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7b12f-ac05-4ca5-9d58-69328c4dcc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:30.775392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7b12f-ac05-4ca5-9d58-69328c4dcc15 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:30.775754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7b12f-ac05-4ca5-9d58-69328c4dcc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:30.776112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7b12f-ac05-4ca5-9d58-69328c4dcc15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:30.784637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7b12f-ac05-4ca5-9d58-69328c4dcc15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:30.786334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7b12f-ac05-4ca5-9d58-69328c4dcc15 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bcd7b12f-ac05-4ca5-9d58-69328c4dcc15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:31:30.805877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:30.807107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:30.812598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:31:30.812598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:31:30.812598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:31:30.812598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:31:30.812598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:31:30.812598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:31:30.812598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:30.812598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:30.812598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 613/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:31:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:30.813872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46672338-e1cd-495a-a519-5a34b6397f4e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:30.929998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46672338-e1cd-495a-a519-5a34b6397f4e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:30.954539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:30.968250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302e41a-d267-48a1-9eb4-bfd10f420cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:30.969051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302e41a-d267-48a1-9eb4-bfd10f420cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:30.969332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302e41a-d267-48a1-9eb4-bfd10f420cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:30.969635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302e41a-d267-48a1-9eb4-bfd10f420cf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:30.970628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302e41a-d267-48a1-9eb4-bfd10f420cf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:30.970991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302e41a-d267-48a1-9eb4-bfd10f420cf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:30.972691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302e41a-d267-48a1-9eb4-bfd10f420cf9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:30.995829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 605/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:31:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:30.996034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7b12f-ac05-4ca5-9d58-69328c4dcc15 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:31.120410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7b12f-ac05-4ca5-9d58-69328c4dcc15 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:31.153321 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:31.160160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1974c-d73b-4d1f-b1ff-37839e2f1140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:31.160938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1974c-d73b-4d1f-b1ff-37839e2f1140 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:31.161187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1974c-d73b-4d1f-b1ff-37839e2f1140 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:31.161467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1974c-d73b-4d1f-b1ff-37839e2f1140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:31.162343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1974c-d73b-4d1f-b1ff-37839e2f1140 None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=cbafea9ed6494a3f9aeba4140a32de4f) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:31.162660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1974c-d73b-4d1f-b1ff-37839e2f1140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:31.164140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1974c-d73b-4d1f-b1ff-37839e2f1140 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:31.179512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 614/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:31:30 2022] GET /identity/v3/role_assignments?user.id=cbafea9ed6494a3f9aeba4140a32de4f&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:33.360649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7ad2c-9f3d-4b4b-b9e6-fb6c957db1c6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:33.361250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7ad2c-9f3d-4b4b-b9e6-fb6c957db1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:33.361250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7ad2c-9f3d-4b4b-b9e6-fb6c957db1c6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:33.362921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 606/1220] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:31:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:31:33.368736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a42502-a8e5-40a8-994d-2d8aed079010 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:33.369170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a42502-a8e5-40a8-994d-2d8aed079010 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:33.369383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a42502-a8e5-40a8-994d-2d8aed079010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:33.389490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-47a42502-a8e5-40a8-994d-2d8aed079010 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:31:33.391024 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-47a42502-a8e5-40a8-994d-2d8aed079010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:31:33.462197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 615/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:31:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:31:33.489055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7ad2c-9f3d-4b4b-b9e6-fb6c957db1c6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:33.605401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7ad2c-9f3d-4b4b-b9e6-fb6c957db1c6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:33.642934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c056f-e481-45af-b38c-ac7916bb37b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:33.643690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c056f-e481-45af-b38c-ac7916bb37b4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:33.643978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c056f-e481-45af-b38c-ac7916bb37b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:33.644259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c056f-e481-45af-b38c-ac7916bb37b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:33.654511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c056f-e481-45af-b38c-ac7916bb37b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3e2c056f-e481-45af-b38c-ac7916bb37b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:31:33.662394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:33.663561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:33.664645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 607/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:31:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:33.670051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a42502-a8e5-40a8-994d-2d8aed079010 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:33.771897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a42502-a8e5-40a8-994d-2d8aed079010 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:33.800600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:33.812999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6b397-bf45-450e-b8ac-aac639c2466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:33.813675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6b397-bf45-450e-b8ac-aac639c2466d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:33.813959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6b397-bf45-450e-b8ac-aac639c2466d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:33.814236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6b397-bf45-450e-b8ac-aac639c2466d None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:33.814995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6b397-bf45-450e-b8ac-aac639c2466d None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:33.815305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6b397-bf45-450e-b8ac-aac639c2466d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:33.816825 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6b397-bf45-450e-b8ac-aac639c2466d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:33.826535 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 616/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:31:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:33.832803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c056f-e481-45af-b38c-ac7916bb37b4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:33.879504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:33.939973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c056f-e481-45af-b38c-ac7916bb37b4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:33.975508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b883dea-8662-4deb-9bab-e41826f506df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:33.976403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b883dea-8662-4deb-9bab-e41826f506df None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:33.976678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b883dea-8662-4deb-9bab-e41826f506df None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:33.976958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b883dea-8662-4deb-9bab-e41826f506df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:33.988216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b883dea-8662-4deb-9bab-e41826f506df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:33.989847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b883dea-8662-4deb-9bab-e41826f506df None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8b883dea-8662-4deb-9bab-e41826f506df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:34.007440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:34.008653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:34.009713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:34.009713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:34.009713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:34.009713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:34.009713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 608/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:31:34.014876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6b397-bf45-450e-b8ac-aac639c2466d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:34.127812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6b397-bf45-450e-b8ac-aac639c2466d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:34.163683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a8b3c-7f2f-4b64-b6fe-2f72d7fc9d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:34.164522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a8b3c-7f2f-4b64-b6fe-2f72d7fc9d59 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:34.164808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a8b3c-7f2f-4b64-b6fe-2f72d7fc9d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:34.165167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a8b3c-7f2f-4b64-b6fe-2f72d7fc9d59 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:34.166055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a8b3c-7f2f-4b64-b6fe-2f72d7fc9d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:34.166368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a8b3c-7f2f-4b64-b6fe-2f72d7fc9d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:34.167937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a8b3c-7f2f-4b64-b6fe-2f72d7fc9d59 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:34.185934 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 617/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:31:34.187383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b883dea-8662-4deb-9bab-e41826f506df None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:34.290081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:34.299129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b883dea-8662-4deb-9bab-e41826f506df None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:34.333299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f91e-8526-4878-89cf-915599055570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:34.334187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f91e-8526-4878-89cf-915599055570 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:34.334470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f91e-8526-4878-89cf-915599055570 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:34.334747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f91e-8526-4878-89cf-915599055570 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:34.342944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47f91e-8526-4878-89cf-915599055570 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:34.344504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47f91e-8526-4878-89cf-915599055570 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2a47f91e-8526-4878-89cf-915599055570 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:31:34.359433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:34.360597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:34.362058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 609/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:34.366707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a8b3c-7f2f-4b64-b6fe-2f72d7fc9d59 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:34.418209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:34.474396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a8b3c-7f2f-4b64-b6fe-2f72d7fc9d59 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:34.512027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ec141-0c24-4cde-8043-95fff3297f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:34.512812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ec141-0c24-4cde-8043-95fff3297f1e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:34.513154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ec141-0c24-4cde-8043-95fff3297f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:34.513443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ec141-0c24-4cde-8043-95fff3297f1e None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:34.514378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ec141-0c24-4cde-8043-95fff3297f1e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:34.514733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ec141-0c24-4cde-8043-95fff3297f1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:34.516281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ec141-0c24-4cde-8043-95fff3297f1e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:34.534086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 618/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 13 20:31:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:34.540063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f91e-8526-4878-89cf-915599055570 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:34.656253 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f91e-8526-4878-89cf-915599055570 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:34.694167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509a470-e5e6-45dd-a71f-ba2794b43dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:34.695278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509a470-e5e6-45dd-a71f-ba2794b43dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:34.695595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509a470-e5e6-45dd-a71f-ba2794b43dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:34.695889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509a470-e5e6-45dd-a71f-ba2794b43dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:34.708021 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509a470-e5e6-45dd-a71f-ba2794b43dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:34.709777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509a470-e5e6-45dd-a71f-ba2794b43dd1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5509a470-e5e6-45dd-a71f-ba2794b43dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:34.729436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:34.730549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:34.731735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:34.731735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:34.731735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:34.731735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:34.731735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 610/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:34.736763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ec141-0c24-4cde-8043-95fff3297f1e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:34.847519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ec141-0c24-4cde-8043-95fff3297f1e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:34.850042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:34.886504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b6d9c-90b8-40f1-9aa4-6d97e3d7732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:34.887220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b6d9c-90b8-40f1-9aa4-6d97e3d7732d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:34.887538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b6d9c-90b8-40f1-9aa4-6d97e3d7732d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:34.887836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b6d9c-90b8-40f1-9aa4-6d97e3d7732d None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:34.888552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b6d9c-90b8-40f1-9aa4-6d97e3d7732d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:34.888860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b6d9c-90b8-40f1-9aa4-6d97e3d7732d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:34.890297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b6d9c-90b8-40f1-9aa4-6d97e3d7732d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:34.902168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 619/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:31:34.908145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509a470-e5e6-45dd-a71f-ba2794b43dd1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:34.919801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:35.023332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509a470-e5e6-45dd-a71f-ba2794b43dd1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:35.059795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3366a-8a38-46eb-b790-81de9fd6d7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:35.060764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3366a-8a38-46eb-b790-81de9fd6d7ba None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:35.061095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3366a-8a38-46eb-b790-81de9fd6d7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:35.061464 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3366a-8a38-46eb-b790-81de9fd6d7ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:35.069461 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3366a-8a38-46eb-b790-81de9fd6d7ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:35.071281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3366a-8a38-46eb-b790-81de9fd6d7ba None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4fe3366a-8a38-46eb-b790-81de9fd6d7ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:31:35.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:35.087591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:35.088621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:31:35.088621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:31:35.088621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:31:35.088621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:31:35.088621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:31:35.088621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:31:35.088621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:35.088621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:35.088621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 611/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:31:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:35.094507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b6d9c-90b8-40f1-9aa4-6d97e3d7732d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:35.203823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b6d9c-90b8-40f1-9aa4-6d97e3d7732d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:35.237935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c94769-34ae-4a66-bd84-272fd743725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:35.238630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c94769-34ae-4a66-bd84-272fd743725c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:35.238913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c94769-34ae-4a66-bd84-272fd743725c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:35.239188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c94769-34ae-4a66-bd84-272fd743725c None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:35.240003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c94769-34ae-4a66-bd84-272fd743725c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:35.240308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c94769-34ae-4a66-bd84-272fd743725c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:35.241760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c94769-34ae-4a66-bd84-272fd743725c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:35.255600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 620/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:31:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:35.261921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3366a-8a38-46eb-b790-81de9fd6d7ba None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:35.329591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:35.366365 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3366a-8a38-46eb-b790-81de9fd6d7ba None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:35.400255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe29706-6e03-44fc-a3ee-474f0f9daff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:35.401170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe29706-6e03-44fc-a3ee-474f0f9daff2 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:35.401454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe29706-6e03-44fc-a3ee-474f0f9daff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:35.401732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe29706-6e03-44fc-a3ee-474f0f9daff2 None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/cbafea9ed6494a3f9aeba4140a32de4f/roles/ee6ccebaaadd458b9d6b4397f2355b3c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:35.420644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe29706-6e03-44fc-a3ee-474f0f9daff2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=cbafea9ed6494a3f9aeba4140a32de4f, role_id=ee6ccebaaadd458b9d6b4397f2355b3c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:35.422196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe29706-6e03-44fc-a3ee-474f0f9daff2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:35.435438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 612/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:31:35 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/cbafea9ed6494a3f9aeba4140a32de4f/roles/ee6ccebaaadd458b9d6b4397f2355b3c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:31:37.523661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24834509-48df-4e01-a9cd-1591bb20e146 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:37.524223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24834509-48df-4e01-a9cd-1591bb20e146 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:37.524223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24834509-48df-4e01-a9cd-1591bb20e146 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:37.525669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 621/1233] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:31:37.531553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de132f-73d3-418a-bbf0-c330b3c80c11 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:37.532183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de132f-73d3-418a-bbf0-c330b3c80c11 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:37.532183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de132f-73d3-418a-bbf0-c330b3c80c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:37.552116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-23de132f-73d3-418a-bbf0-c330b3c80c11 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:31:37.553722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-23de132f-73d3-418a-bbf0-c330b3c80c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:31:37.628716 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 613/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:31:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:31:37.652703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24834509-48df-4e01-a9cd-1591bb20e146 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:37.680203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:37.768672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24834509-48df-4e01-a9cd-1591bb20e146 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:37.809025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b6dfe-b10d-4a58-82c5-4df7c2ee46d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:37.809761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b6dfe-b10d-4a58-82c5-4df7c2ee46d8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:37.810047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b6dfe-b10d-4a58-82c5-4df7c2ee46d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:37.810333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b6dfe-b10d-4a58-82c5-4df7c2ee46d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:37.822112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b6dfe-b10d-4a58-82c5-4df7c2ee46d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a34b6dfe-b10d-4a58-82c5-4df7c2ee46d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:31:37.829878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:37.831311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:37.832583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 622/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 13 20:31:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:37.837827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de132f-73d3-418a-bbf0-c330b3c80c11 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:37.893144 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:37.951094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de132f-73d3-418a-bbf0-c330b3c80c11 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:37.987300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2b93a-5170-4af5-b5a2-17d973002108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:37.988008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2b93a-5170-4af5-b5a2-17d973002108 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:37.988591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2b93a-5170-4af5-b5a2-17d973002108 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:37.989755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2b93a-5170-4af5-b5a2-17d973002108 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:37.990503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2b93a-5170-4af5-b5a2-17d973002108 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:37.990818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2b93a-5170-4af5-b5a2-17d973002108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:37.992264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2b93a-5170-4af5-b5a2-17d973002108 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:38.000882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 614/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 13 20:31:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:38.006730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b6dfe-b10d-4a58-82c5-4df7c2ee46d8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:38.094522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:38.115630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b6dfe-b10d-4a58-82c5-4df7c2ee46d8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:38.154345 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054e9bf-b9ac-4cb7-9cc7-c77e229263b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:38.155156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054e9bf-b9ac-4cb7-9cc7-c77e229263b4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:38.155443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054e9bf-b9ac-4cb7-9cc7-c77e229263b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:38.155750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054e9bf-b9ac-4cb7-9cc7-c77e229263b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:38.168653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054e9bf-b9ac-4cb7-9cc7-c77e229263b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:38.170612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054e9bf-b9ac-4cb7-9cc7-c77e229263b4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-6054e9bf-b9ac-4cb7-9cc7-c77e229263b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:38.190416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:38.191451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:38.198248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:38.198248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:38.198248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:38.198248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:38.198248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 623/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:31:38.198783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2b93a-5170-4af5-b5a2-17d973002108 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:38.306201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2b93a-5170-4af5-b5a2-17d973002108 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:38.315803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:38.342286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253024d-77e0-4de5-9674-289513fdb999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:38.342956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253024d-77e0-4de5-9674-289513fdb999 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:38.343248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253024d-77e0-4de5-9674-289513fdb999 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:38.343565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253024d-77e0-4de5-9674-289513fdb999 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:38.344285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253024d-77e0-4de5-9674-289513fdb999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:38.344607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253024d-77e0-4de5-9674-289513fdb999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:38.345991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253024d-77e0-4de5-9674-289513fdb999 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:38.358549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 615/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:38.364251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054e9bf-b9ac-4cb7-9cc7-c77e229263b4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:38.471192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054e9bf-b9ac-4cb7-9cc7-c77e229263b4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:38.506803 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f3f0e-201a-4b1b-b5e7-d76be8447467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:38.507653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f3f0e-201a-4b1b-b5e7-d76be8447467 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:38.507943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f3f0e-201a-4b1b-b5e7-d76be8447467 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:38.508221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f3f0e-201a-4b1b-b5e7-d76be8447467 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:38.510953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:38.516872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f3f0e-201a-4b1b-b5e7-d76be8447467 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:38.518306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f3f0e-201a-4b1b-b5e7-d76be8447467 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-887f3f0e-201a-4b1b-b5e7-d76be8447467 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:31:38.532922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:38.534273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:38.535298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 624/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:38.540079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253024d-77e0-4de5-9674-289513fdb999 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:38.646463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253024d-77e0-4de5-9674-289513fdb999 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:38.686646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cdaba-ca29-4880-99aa-173ef887083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:38.687409 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cdaba-ca29-4880-99aa-173ef887083c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:38.687784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cdaba-ca29-4880-99aa-173ef887083c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:38.688222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cdaba-ca29-4880-99aa-173ef887083c None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:38.689156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821cdaba-ca29-4880-99aa-173ef887083c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:38.689566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821cdaba-ca29-4880-99aa-173ef887083c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:38.691114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821cdaba-ca29-4880-99aa-173ef887083c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:38.719616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 616/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 13 20:31:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:38.725987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f3f0e-201a-4b1b-b5e7-d76be8447467 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:38.856267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f3f0e-201a-4b1b-b5e7-d76be8447467 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:38.894862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56e26c-e0a7-434a-90f8-016cccd6a2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:38.895560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56e26c-e0a7-434a-90f8-016cccd6a2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:38.895914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56e26c-e0a7-434a-90f8-016cccd6a2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:38.896289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56e26c-e0a7-434a-90f8-016cccd6a2b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:38.966973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:38.975222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56e26c-e0a7-434a-90f8-016cccd6a2b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:38.976753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56e26c-e0a7-434a-90f8-016cccd6a2b4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:39.091493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 625/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:31:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:31:39.097467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cdaba-ca29-4880-99aa-173ef887083c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:39.130981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:39.215700 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cdaba-ca29-4880-99aa-173ef887083c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:39.255578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e500ac9-b47c-4bfc-addf-2c6816316c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:39.256545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e500ac9-b47c-4bfc-addf-2c6816316c29 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:39.256924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e500ac9-b47c-4bfc-addf-2c6816316c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:39.257309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e500ac9-b47c-4bfc-addf-2c6816316c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:39.269381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e500ac9-b47c-4bfc-addf-2c6816316c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:39.271238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e500ac9-b47c-4bfc-addf-2c6816316c29 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9e500ac9-b47c-4bfc-addf-2c6816316c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:39.291399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:39.292390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:39.293348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:39.293348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:39.293348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:39.293348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:39.293348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 617/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:39.299214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56e26c-e0a7-434a-90f8-016cccd6a2b4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:39.408311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:39.415314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56e26c-e0a7-434a-90f8-016cccd6a2b4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:39.453988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d89d77-742b-4101-bcaa-145129fca74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:39.454676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d89d77-742b-4101-bcaa-145129fca74a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:39.455019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d89d77-742b-4101-bcaa-145129fca74a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:39.455288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d89d77-742b-4101-bcaa-145129fca74a None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:39.456143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d89d77-742b-4101-bcaa-145129fca74a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:39.456460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d89d77-742b-4101-bcaa-145129fca74a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:39.457982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d89d77-742b-4101-bcaa-145129fca74a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:39.470348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 626/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:39.476421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e500ac9-b47c-4bfc-addf-2c6816316c29 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:39.561215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:39.585062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e500ac9-b47c-4bfc-addf-2c6816316c29 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:39.624731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3be3e2-1148-43bc-b7c2-f6512d360aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:39.625546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3be3e2-1148-43bc-b7c2-f6512d360aba None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:39.625835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3be3e2-1148-43bc-b7c2-f6512d360aba None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:39.626116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3be3e2-1148-43bc-b7c2-f6512d360aba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:39.633285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3be3e2-1148-43bc-b7c2-f6512d360aba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:39.634880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3be3e2-1148-43bc-b7c2-f6512d360aba None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-cc3be3e2-1148-43bc-b7c2-f6512d360aba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:31:39.649596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:39.650719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:39.652012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:31:39.652012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:31:39.652012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:31:39.652012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:31:39.652012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:31:39.652012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:31:39.652012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:39.652012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:39.652012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 618/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:31:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:39.656901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d89d77-742b-4101-bcaa-145129fca74a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:39.764534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d89d77-742b-4101-bcaa-145129fca74a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:39.809788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf9c56-c722-423a-a9a6-df9bdfdbd728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:39.810636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf9c56-c722-423a-a9a6-df9bdfdbd728 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:39.810868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf9c56-c722-423a-a9a6-df9bdfdbd728 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:39.811172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf9c56-c722-423a-a9a6-df9bdfdbd728 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:39.812410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf9c56-c722-423a-a9a6-df9bdfdbd728 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:39.812410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf9c56-c722-423a-a9a6-df9bdfdbd728 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:39.813954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf9c56-c722-423a-a9a6-df9bdfdbd728 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:39.829670 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 627/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:31:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:39.838757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3be3e2-1148-43bc-b7c2-f6512d360aba None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:39.950653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3be3e2-1148-43bc-b7c2-f6512d360aba None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:39.988275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b9d97-332a-4cf4-8576-860119bd25ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:39.988996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b9d97-332a-4cf4-8576-860119bd25ad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:39.989362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b9d97-332a-4cf4-8576-860119bd25ad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:39.989665 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b9d97-332a-4cf4-8576-860119bd25ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:39.990515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b9d97-332a-4cf4-8576-860119bd25ad None admin] RBAC: Adding query filter params (role.id=ee6ccebaaadd458b9d6b4397f2355b3c, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=cbafea9ed6494a3f9aeba4140a32de4f) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:39.990852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b9d97-332a-4cf4-8576-860119bd25ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:39.992320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b9d97-332a-4cf4-8576-860119bd25ad None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:39.999097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 619/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:31:39 2022] GET /identity/v3/role_assignments?user.id=cbafea9ed6494a3f9aeba4140a32de4f&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=ee6ccebaaadd458b9d6b4397f2355b3c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:42.153837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b5e9c-3348-4c68-969f-e93efabea15a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:42.154432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b5e9c-3348-4c68-969f-e93efabea15a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:42.154432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b5e9c-3348-4c68-969f-e93efabea15a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:42.155782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 628/1247] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:31:42.161094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce3321-241b-428d-a215-fcd0f1f0d9ad None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:42.161576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce3321-241b-428d-a215-fcd0f1f0d9ad None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:42.161672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce3321-241b-428d-a215-fcd0f1f0d9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:42.168318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:42.183674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4bce3321-241b-428d-a215-fcd0f1f0d9ad None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:31:42.184874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4bce3321-241b-428d-a215-fcd0f1f0d9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:31:42.260022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 620/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:31:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:31:42.283726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b5e9c-3348-4c68-969f-e93efabea15a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:42.303267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:42.403055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b5e9c-3348-4c68-969f-e93efabea15a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:42.441177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a7989-1aa7-42f8-81bb-527d34d82f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:42.442046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a7989-1aa7-42f8-81bb-527d34d82f13 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:42.442291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a7989-1aa7-42f8-81bb-527d34d82f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:42.442609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a7989-1aa7-42f8-81bb-527d34d82f13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:42.455088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a7989-1aa7-42f8-81bb-527d34d82f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-418a7989-1aa7-42f8-81bb-527d34d82f13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:31:42.463672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:42.464838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:42.466106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 629/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:31:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:42.471245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce3321-241b-428d-a215-fcd0f1f0d9ad None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:42.577632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce3321-241b-428d-a215-fcd0f1f0d9ad None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:42.612589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:42.618345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d11ff-a0ec-444d-8d73-fe14d28dc277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:42.619148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d11ff-a0ec-444d-8d73-fe14d28dc277 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:42.619470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d11ff-a0ec-444d-8d73-fe14d28dc277 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:42.619837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d11ff-a0ec-444d-8d73-fe14d28dc277 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:42.620591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d11ff-a0ec-444d-8d73-fe14d28dc277 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:42.620898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d11ff-a0ec-444d-8d73-fe14d28dc277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:42.622506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d11ff-a0ec-444d-8d73-fe14d28dc277 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:42.631859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 621/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:31:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:42.638180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a7989-1aa7-42f8-81bb-527d34d82f13 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:42.725393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:42.751065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a7989-1aa7-42f8-81bb-527d34d82f13 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:42.790123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a5c1a-ed95-4ffd-abcf-f3fdce38dd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:42.790934 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a5c1a-ed95-4ffd-abcf-f3fdce38dd13 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:42.791443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a5c1a-ed95-4ffd-abcf-f3fdce38dd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:42.791696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a5c1a-ed95-4ffd-abcf-f3fdce38dd13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:42.805671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a5c1a-ed95-4ffd-abcf-f3fdce38dd13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:42.807834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a5c1a-ed95-4ffd-abcf-f3fdce38dd13 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-802a5c1a-ed95-4ffd-abcf-f3fdce38dd13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:42.831080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:42.832145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:42.833027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:42.833027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:42.833027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:42.833027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:42.839376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 630/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:42.839555 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d11ff-a0ec-444d-8d73-fe14d28dc277 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:42.967957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d11ff-a0ec-444d-8d73-fe14d28dc277 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:43.005798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf38e-df69-406d-816d-84ec44f324ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:43.006394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf38e-df69-406d-816d-84ec44f324ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:43.006682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf38e-df69-406d-816d-84ec44f324ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:43.006962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf38e-df69-406d-816d-84ec44f324ac None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:43.007727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf38e-df69-406d-816d-84ec44f324ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:43.008038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf38e-df69-406d-816d-84ec44f324ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:43.009581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf38e-df69-406d-816d-84ec44f324ac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:43.032632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 622/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:43.034345 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a5c1a-ed95-4ffd-abcf-f3fdce38dd13 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:43.152403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a5c1a-ed95-4ffd-abcf-f3fdce38dd13 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:43.181334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:43.189574 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cccc82-57ab-425e-a359-8978f9e93091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:43.190389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cccc82-57ab-425e-a359-8978f9e93091 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:43.190667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cccc82-57ab-425e-a359-8978f9e93091 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:43.190946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cccc82-57ab-425e-a359-8978f9e93091 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:43.200396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cccc82-57ab-425e-a359-8978f9e93091 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:43.201982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cccc82-57ab-425e-a359-8978f9e93091 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c8cccc82-57ab-425e-a359-8978f9e93091 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:31:43.217411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:43.218749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:43.220145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 631/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:43.225154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf38e-df69-406d-816d-84ec44f324ac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:43.284805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:43.333526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf38e-df69-406d-816d-84ec44f324ac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:43.371552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5ee9d-c474-43f7-a464-b0f58b2692b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:43.372275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5ee9d-c474-43f7-a464-b0f58b2692b0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:43.372595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5ee9d-c474-43f7-a464-b0f58b2692b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:43.372874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5ee9d-c474-43f7-a464-b0f58b2692b0 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:43.373799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5ee9d-c474-43f7-a464-b0f58b2692b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:43.374114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5ee9d-c474-43f7-a464-b0f58b2692b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:43.375669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5ee9d-c474-43f7-a464-b0f58b2692b0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:43.394334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 623/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 13 20:31:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:43.400244 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cccc82-57ab-425e-a359-8978f9e93091 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:43.517800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cccc82-57ab-425e-a359-8978f9e93091 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:43.554026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dda590-0c1c-4e48-ab8c-631aeaea7810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:43.554641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dda590-0c1c-4e48-ab8c-631aeaea7810 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:43.554989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dda590-0c1c-4e48-ab8c-631aeaea7810 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:43.555283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dda590-0c1c-4e48-ab8c-631aeaea7810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:43.619713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:43.632882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dda590-0c1c-4e48-ab8c-631aeaea7810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:43.634467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dda590-0c1c-4e48-ab8c-631aeaea7810 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:43.748035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 632/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:31:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:31:43.754447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5ee9d-c474-43f7-a464-b0f58b2692b0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:43.869141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5ee9d-c474-43f7-a464-b0f58b2692b0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:43.884097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:43.907114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e32cd0-e0de-47ae-a4c4-f28cdf2c7c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:43.908084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e32cd0-e0de-47ae-a4c4-f28cdf2c7c21 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:43.908418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e32cd0-e0de-47ae-a4c4-f28cdf2c7c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:43.908720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e32cd0-e0de-47ae-a4c4-f28cdf2c7c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:43.919663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e32cd0-e0de-47ae-a4c4-f28cdf2c7c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:43.921376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e32cd0-e0de-47ae-a4c4-f28cdf2c7c21 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-43e32cd0-e0de-47ae-a4c4-f28cdf2c7c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:43.939282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:43.940390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:43.941340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:43.941340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:43.941340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:43.941340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:43.941340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 624/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:43.946968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dda590-0c1c-4e48-ab8c-631aeaea7810 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:44.051554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:44.068647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dda590-0c1c-4e48-ab8c-631aeaea7810 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:44.108771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e84d64-050e-47d4-a125-94e86867a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:44.109455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e84d64-050e-47d4-a125-94e86867a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:44.109740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e84d64-050e-47d4-a125-94e86867a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:44.110031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e84d64-050e-47d4-a125-94e86867a3c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:44.110758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e84d64-050e-47d4-a125-94e86867a3c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:44.111070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e84d64-050e-47d4-a125-94e86867a3c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:44.112607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e84d64-050e-47d4-a125-94e86867a3c8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:44.125102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 633/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:44.132126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e32cd0-e0de-47ae-a4c4-f28cdf2c7c21 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:44.243041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e32cd0-e0de-47ae-a4c4-f28cdf2c7c21 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:44.277475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf9cae-2703-44c3-adc4-6c651b93f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:44.278271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf9cae-2703-44c3-adc4-6c651b93f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:44.278562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf9cae-2703-44c3-adc4-6c651b93f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:44.278843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf9cae-2703-44c3-adc4-6c651b93f7e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:44.285739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf9cae-2703-44c3-adc4-6c651b93f7e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:44.287213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf9cae-2703-44c3-adc4-6c651b93f7e6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-cdcf9cae-2703-44c3-adc4-6c651b93f7e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:31:44.300999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:44.302143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:44.308594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:31:44.308594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:31:44.308594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:31:44.308594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:31:44.308594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:31:44.308594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:31:44.308594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:44.308594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:44.308594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 625/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:31:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:31:44.309189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e84d64-050e-47d4-a125-94e86867a3c8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:44.412602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e84d64-050e-47d4-a125-94e86867a3c8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:44.446561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490b2e7-28c2-467e-8dbf-fcf6c5c225da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:44.447224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490b2e7-28c2-467e-8dbf-fcf6c5c225da None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:44.447505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490b2e7-28c2-467e-8dbf-fcf6c5c225da None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:44.447808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490b2e7-28c2-467e-8dbf-fcf6c5c225da None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:44.448595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490b2e7-28c2-467e-8dbf-fcf6c5c225da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:44.448956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490b2e7-28c2-467e-8dbf-fcf6c5c225da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:44.450444 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490b2e7-28c2-467e-8dbf-fcf6c5c225da None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:44.463160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 634/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:31:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:31:44.469500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf9cae-2703-44c3-adc4-6c651b93f7e6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:44.500203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:44.582743 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf9cae-2703-44c3-adc4-6c651b93f7e6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:44.617821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c885c-5ccd-42ee-a677-c78f5023bb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:44.618484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c885c-5ccd-42ee-a677-c78f5023bb58 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:44.618776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c885c-5ccd-42ee-a677-c78f5023bb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:44.619057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c885c-5ccd-42ee-a677-c78f5023bb58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:44.620054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c885c-5ccd-42ee-a677-c78f5023bb58 None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=cbafea9ed6494a3f9aeba4140a32de4f) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:44.620394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c885c-5ccd-42ee-a677-c78f5023bb58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:44.621836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c885c-5ccd-42ee-a677-c78f5023bb58 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:44.627865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 626/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:31:44 2022] GET /identity/v3/role_assignments?user.id=cbafea9ed6494a3f9aeba4140a32de4f&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:31:46.645805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4f11-ad0d-4280-8bda-a02a0afd703b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:46.646689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4f11-ad0d-4280-8bda-a02a0afd703b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:46.646689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4f11-ad0d-4280-8bda-a02a0afd703b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:46.648836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 635/1261] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:31:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:31:46.655230 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f96d7f-efc8-4cac-8970-d9ac5b3ab338 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:46.655981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f96d7f-efc8-4cac-8970-d9ac5b3ab338 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:46.656203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f96d7f-efc8-4cac-8970-d9ac5b3ab338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:46.674795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b3f96d7f-efc8-4cac-8970-d9ac5b3ab338 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:31:46.676024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b3f96d7f-efc8-4cac-8970-d9ac5b3ab338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:31:46.744449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 627/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:31:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:31:46.770486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad4f11-ad0d-4280-8bda-a02a0afd703b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:46.782798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:46.893263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad4f11-ad0d-4280-8bda-a02a0afd703b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:46.927868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99c6fd-52ea-4eb9-83cb-236852d3f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:46.928601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99c6fd-52ea-4eb9-83cb-236852d3f98d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:46.928886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99c6fd-52ea-4eb9-83cb-236852d3f98d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:46.929165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99c6fd-52ea-4eb9-83cb-236852d3f98d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:46.939862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99c6fd-52ea-4eb9-83cb-236852d3f98d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2c99c6fd-52ea-4eb9-83cb-236852d3f98d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:31:46.947897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:46.948960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:46.950047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 636/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:31:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:46.956028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f96d7f-efc8-4cac-8970-d9ac5b3ab338 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:46.975995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:47.068896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f96d7f-efc8-4cac-8970-d9ac5b3ab338 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:47.106674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495839e-a965-4ac9-a9d9-574ea2fbca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:47.107337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495839e-a965-4ac9-a9d9-574ea2fbca25 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:47.107647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495839e-a965-4ac9-a9d9-574ea2fbca25 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:47.107927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495839e-a965-4ac9-a9d9-574ea2fbca25 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:47.108676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495839e-a965-4ac9-a9d9-574ea2fbca25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:47.108990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495839e-a965-4ac9-a9d9-574ea2fbca25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:47.110374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495839e-a965-4ac9-a9d9-574ea2fbca25 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:47.118595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 628/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:31:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:47.126570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99c6fd-52ea-4eb9-83cb-236852d3f98d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:47.209820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:47.242586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99c6fd-52ea-4eb9-83cb-236852d3f98d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:47.277001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d0026-45e1-4dbe-a546-e0502a311552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:47.277785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d0026-45e1-4dbe-a546-e0502a311552 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:47.278070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d0026-45e1-4dbe-a546-e0502a311552 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:47.278347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d0026-45e1-4dbe-a546-e0502a311552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:47.289272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d0026-45e1-4dbe-a546-e0502a311552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:47.290717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d0026-45e1-4dbe-a546-e0502a311552 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-356d0026-45e1-4dbe-a546-e0502a311552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:47.308228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:47.309236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:47.315329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:47.315329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:47.315329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:47.315329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:47.315329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 637/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:47.317424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495839e-a965-4ac9-a9d9-574ea2fbca25 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:47.401171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:47.421466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495839e-a965-4ac9-a9d9-574ea2fbca25 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:47.455004 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40040dbb-f614-4b8c-81fb-851e87e01b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:47.455688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40040dbb-f614-4b8c-81fb-851e87e01b80 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:47.455971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40040dbb-f614-4b8c-81fb-851e87e01b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:47.456243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40040dbb-f614-4b8c-81fb-851e87e01b80 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:47.456957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40040dbb-f614-4b8c-81fb-851e87e01b80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:47.457259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40040dbb-f614-4b8c-81fb-851e87e01b80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:47.458636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40040dbb-f614-4b8c-81fb-851e87e01b80 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:47.470018 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 629/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:47.476156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d0026-45e1-4dbe-a546-e0502a311552 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:47.579501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d0026-45e1-4dbe-a546-e0502a311552 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:47.599766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:47.616744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46eddd3-8b7c-4a5c-bdd1-b3c4137e7313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:47.617628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46eddd3-8b7c-4a5c-bdd1-b3c4137e7313 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:47.617905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46eddd3-8b7c-4a5c-bdd1-b3c4137e7313 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:47.618182 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46eddd3-8b7c-4a5c-bdd1-b3c4137e7313 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:47.626012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46eddd3-8b7c-4a5c-bdd1-b3c4137e7313 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:47.627443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46eddd3-8b7c-4a5c-bdd1-b3c4137e7313 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a46eddd3-8b7c-4a5c-bdd1-b3c4137e7313 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:31:47.641792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:47.642917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:47.644203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 638/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:47.650884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40040dbb-f614-4b8c-81fb-851e87e01b80 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:47.754802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40040dbb-f614-4b8c-81fb-851e87e01b80 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:47.790885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2746e8-9f9b-49b0-89b3-419ee623dac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:47.791564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2746e8-9f9b-49b0-89b3-419ee623dac5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:47.791855 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2746e8-9f9b-49b0-89b3-419ee623dac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:47.792130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2746e8-9f9b-49b0-89b3-419ee623dac5 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:47.792965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2746e8-9f9b-49b0-89b3-419ee623dac5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:47.793279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2746e8-9f9b-49b0-89b3-419ee623dac5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:47.794719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2746e8-9f9b-49b0-89b3-419ee623dac5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:47.803773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:47.812981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 630/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 13 20:31:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:47.818722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46eddd3-8b7c-4a5c-bdd1-b3c4137e7313 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:47.923131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46eddd3-8b7c-4a5c-bdd1-b3c4137e7313 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:47.959093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f8394-e951-48e6-99c8-6b2eb5544bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:47.959925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f8394-e951-48e6-99c8-6b2eb5544bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:47.960214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f8394-e951-48e6-99c8-6b2eb5544bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:47.960498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f8394-e951-48e6-99c8-6b2eb5544bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:47.971618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f8394-e951-48e6-99c8-6b2eb5544bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:47.973166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f8394-e951-48e6-99c8-6b2eb5544bb9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a57f8394-e951-48e6-99c8-6b2eb5544bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:47.990418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:47.991402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:47.992731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:47.992731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:47.992731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:47.992731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:47.992731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 639/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:47.999383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2746e8-9f9b-49b0-89b3-419ee623dac5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:48.107106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2746e8-9f9b-49b0-89b3-419ee623dac5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:48.149322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b98d6-1ecd-4b72-a35f-6d9328602318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:48.150255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b98d6-1ecd-4b72-a35f-6d9328602318 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:48.150648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b98d6-1ecd-4b72-a35f-6d9328602318 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:48.151031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b98d6-1ecd-4b72-a35f-6d9328602318 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:48.152205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b98d6-1ecd-4b72-a35f-6d9328602318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:48.152642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b98d6-1ecd-4b72-a35f-6d9328602318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:48.154796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b98d6-1ecd-4b72-a35f-6d9328602318 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:48.168199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 631/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:48.173967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f8394-e951-48e6-99c8-6b2eb5544bb9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:48.208910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:48.285157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f8394-e951-48e6-99c8-6b2eb5544bb9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:48.320611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd82ba-656f-44a2-b0c3-6e8165e7c4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:48.321326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd82ba-656f-44a2-b0c3-6e8165e7c4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:48.321608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd82ba-656f-44a2-b0c3-6e8165e7c4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:48.321888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd82ba-656f-44a2-b0c3-6e8165e7c4f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:48.328864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd82ba-656f-44a2-b0c3-6e8165e7c4f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:48.330311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd82ba-656f-44a2-b0c3-6e8165e7c4f3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d1dd82ba-656f-44a2-b0c3-6e8165e7c4f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:31:48.344802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:48.345808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:48.349469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:31:48.349469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:31:48.349469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:31:48.349469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:31:48.349469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:31:48.349469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:31:48.349469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:48.349469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:48.349469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 640/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:31:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:31:48.352366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b98d6-1ecd-4b72-a35f-6d9328602318 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:48.410054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:48.453282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b98d6-1ecd-4b72-a35f-6d9328602318 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:48.488482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c92d1-c748-4e95-a4d6-52b2d1bd7e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:48.489130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c92d1-c748-4e95-a4d6-52b2d1bd7e74 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:48.489419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c92d1-c748-4e95-a4d6-52b2d1bd7e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:48.489692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c92d1-c748-4e95-a4d6-52b2d1bd7e74 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:48.490480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c92d1-c748-4e95-a4d6-52b2d1bd7e74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:48.490789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c92d1-c748-4e95-a4d6-52b2d1bd7e74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:48.492267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c92d1-c748-4e95-a4d6-52b2d1bd7e74 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:48.504171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 632/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:48.513049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd82ba-656f-44a2-b0c3-6e8165e7c4f3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:48.606146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:48.620720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd82ba-656f-44a2-b0c3-6e8165e7c4f3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:48.665229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72dbe7-b1fe-426f-8ac9-d994b8ce32c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:48.666111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72dbe7-b1fe-426f-8ac9-d994b8ce32c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:48.666392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72dbe7-b1fe-426f-8ac9-d994b8ce32c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:48.666669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72dbe7-b1fe-426f-8ac9-d994b8ce32c8 None admin] PATH_INFO: `/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/cbafea9ed6494a3f9aeba4140a32de4f/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:48.668842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72dbe7-b1fe-426f-8ac9-d994b8ce32c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef928cdf3dcd4d7392f3749bf55bec06, user_id=cbafea9ed6494a3f9aeba4140a32de4f, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:48.670282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72dbe7-b1fe-426f-8ac9-d994b8ce32c8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:48.685671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 641/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 13 20:31:48 2022] PUT /identity/v3/projects/ef928cdf3dcd4d7392f3749bf55bec06/users/cbafea9ed6494a3f9aeba4140a32de4f/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:31:50.683401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d894a-b32e-4eb6-99fd-fd8f236ce649 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:50.683954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d894a-b32e-4eb6-99fd-fd8f236ce649 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:50.684076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d894a-b32e-4eb6-99fd-fd8f236ce649 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:50.686129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 633/1274] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:31:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:31:50.695123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d5bfb-6190-4bfd-bfd5-d4d261fc7dd9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:50.695538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d5bfb-6190-4bfd-bfd5-d4d261fc7dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:50.695810 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d5bfb-6190-4bfd-bfd5-d4d261fc7dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:50.723428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b82d5bfb-6190-4bfd-bfd5-d4d261fc7dd9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:31:50.725259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b82d5bfb-6190-4bfd-bfd5-d4d261fc7dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:31:50.801319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 642/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:31:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:31:50.829235 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d894a-b32e-4eb6-99fd-fd8f236ce649 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:50.935264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d894a-b32e-4eb6-99fd-fd8f236ce649 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:50.952973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:50.971243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5783f-3b1a-4703-af43-8f32d9b702ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:50.972162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5783f-3b1a-4703-af43-8f32d9b702ad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:50.972443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5783f-3b1a-4703-af43-8f32d9b702ad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:50.972714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5783f-3b1a-4703-af43-8f32d9b702ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:50.983504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5783f-3b1a-4703-af43-8f32d9b702ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-43a5783f-3b1a-4703-af43-8f32d9b702ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:31:50.990988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:50.992074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:50.993171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 634/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 13 20:31:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:50.998225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d5bfb-6190-4bfd-bfd5-d4d261fc7dd9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:51.099204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:51.119706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d5bfb-6190-4bfd-bfd5-d4d261fc7dd9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:51.154718 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2beb06d-4cbe-42aa-98ee-ab951ad84f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:51.155375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2beb06d-4cbe-42aa-98ee-ab951ad84f95 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:51.155688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2beb06d-4cbe-42aa-98ee-ab951ad84f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:51.155961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2beb06d-4cbe-42aa-98ee-ab951ad84f95 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:51.156701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2beb06d-4cbe-42aa-98ee-ab951ad84f95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:51.157013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2beb06d-4cbe-42aa-98ee-ab951ad84f95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:51.158379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2beb06d-4cbe-42aa-98ee-ab951ad84f95 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:51.166438 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 643/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 13 20:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:51.172449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5783f-3b1a-4703-af43-8f32d9b702ad None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:51.278948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5783f-3b1a-4703-af43-8f32d9b702ad None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:51.313559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550063cf-ba02-44fc-9a17-50a92fc9853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:51.314335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550063cf-ba02-44fc-9a17-50a92fc9853a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:51.314613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550063cf-ba02-44fc-9a17-50a92fc9853a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:51.314892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550063cf-ba02-44fc-9a17-50a92fc9853a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:51.326108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550063cf-ba02-44fc-9a17-50a92fc9853a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:51.327613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550063cf-ba02-44fc-9a17-50a92fc9853a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-550063cf-ba02-44fc-9a17-50a92fc9853a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:51.344882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:51.345871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:51.346778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:51.346778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:51.346778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:51.346778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:51.346778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 635/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:51.354653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2beb06d-4cbe-42aa-98ee-ab951ad84f95 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:51.472945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2beb06d-4cbe-42aa-98ee-ab951ad84f95 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:51.508222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc3e22-8b3e-4b28-b80a-2fcb0b76c49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:51.508874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc3e22-8b3e-4b28-b80a-2fcb0b76c49c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:51.509156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc3e22-8b3e-4b28-b80a-2fcb0b76c49c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:51.509445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc3e22-8b3e-4b28-b80a-2fcb0b76c49c None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:51.510161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc3e22-8b3e-4b28-b80a-2fcb0b76c49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:51.510474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc3e22-8b3e-4b28-b80a-2fcb0b76c49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:51.511898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc3e22-8b3e-4b28-b80a-2fcb0b76c49c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:51.523773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 644/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:51.532757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550063cf-ba02-44fc-9a17-50a92fc9853a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:51.572825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:51.649100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550063cf-ba02-44fc-9a17-50a92fc9853a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:51.683779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cba5ac-2357-4035-bc6d-c2bfb0cf4c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:51.684574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cba5ac-2357-4035-bc6d-c2bfb0cf4c21 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:51.684858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cba5ac-2357-4035-bc6d-c2bfb0cf4c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:51.685134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cba5ac-2357-4035-bc6d-c2bfb0cf4c21 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:51.692455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cba5ac-2357-4035-bc6d-c2bfb0cf4c21 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:51.693906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cba5ac-2357-4035-bc6d-c2bfb0cf4c21 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-99cba5ac-2357-4035-bc6d-c2bfb0cf4c21 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:31:51.709212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:51.710251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:51.711399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 636/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:51.717018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc3e22-8b3e-4b28-b80a-2fcb0b76c49c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:51.722836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:51.833970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc3e22-8b3e-4b28-b80a-2fcb0b76c49c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:51.868942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265026c-b9f3-49b0-8b74-6f1b5c4cd242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:51.869612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265026c-b9f3-49b0-8b74-6f1b5c4cd242 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:51.869899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265026c-b9f3-49b0-8b74-6f1b5c4cd242 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:51.870178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265026c-b9f3-49b0-8b74-6f1b5c4cd242 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:51.871014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265026c-b9f3-49b0-8b74-6f1b5c4cd242 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:51.871327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265026c-b9f3-49b0-8b74-6f1b5c4cd242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:51.872810 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265026c-b9f3-49b0-8b74-6f1b5c4cd242 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:51.890047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 645/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 13 20:31:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:51.896849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cba5ac-2357-4035-bc6d-c2bfb0cf4c21 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:51.998050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:52.016603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cba5ac-2357-4035-bc6d-c2bfb0cf4c21 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:52.057999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8fd9d-b442-449a-b338-0ce58f910eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:52.058673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8fd9d-b442-449a-b338-0ce58f910eaa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:52.058940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8fd9d-b442-449a-b338-0ce58f910eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:52.059219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8fd9d-b442-449a-b338-0ce58f910eaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:52.129721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8fd9d-b442-449a-b338-0ce58f910eaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:52.131115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8fd9d-b442-449a-b338-0ce58f910eaa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:52.229333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:52.240302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 637/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 13 20:31:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:31:52.246336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265026c-b9f3-49b0-8b74-6f1b5c4cd242 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:52.312406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:52.356920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265026c-b9f3-49b0-8b74-6f1b5c4cd242 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:52.393407 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949469ba-b19c-4fe5-ae81-f83c569aace4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:52.394215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949469ba-b19c-4fe5-ae81-f83c569aace4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:52.394501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949469ba-b19c-4fe5-ae81-f83c569aace4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:52.394779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949469ba-b19c-4fe5-ae81-f83c569aace4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:52.405328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949469ba-b19c-4fe5-ae81-f83c569aace4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:52.406777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949469ba-b19c-4fe5-ae81-f83c569aace4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-949469ba-b19c-4fe5-ae81-f83c569aace4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:31:52.424437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:52.425482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:52.426671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:31:52.426671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:31:52.426671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:31:52.426671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:52.426671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 646/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:52.431496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8fd9d-b442-449a-b338-0ce58f910eaa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:52.534459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8fd9d-b442-449a-b338-0ce58f910eaa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:52.575983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e60f1-ee0a-46cb-a426-fc51c88ca51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:52.576642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e60f1-ee0a-46cb-a426-fc51c88ca51c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:52.576933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e60f1-ee0a-46cb-a426-fc51c88ca51c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:52.577214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e60f1-ee0a-46cb-a426-fc51c88ca51c None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:52.577937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e60f1-ee0a-46cb-a426-fc51c88ca51c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:52.578251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e60f1-ee0a-46cb-a426-fc51c88ca51c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:52.579686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e60f1-ee0a-46cb-a426-fc51c88ca51c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:52.591296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 638/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 13 20:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:31:52.596994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949469ba-b19c-4fe5-ae81-f83c569aace4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:52.713788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949469ba-b19c-4fe5-ae81-f83c569aace4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:52.729803 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:52.754388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65190b-07a2-4345-ad1f-2066c66f57fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:52.755174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65190b-07a2-4345-ad1f-2066c66f57fb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:52.755503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65190b-07a2-4345-ad1f-2066c66f57fb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:52.755766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65190b-07a2-4345-ad1f-2066c66f57fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:52.762657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65190b-07a2-4345-ad1f-2066c66f57fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:52.764139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65190b-07a2-4345-ad1f-2066c66f57fb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9e65190b-07a2-4345-ad1f-2066c66f57fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:31:52.778179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:52.779169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:52.780224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:31:52.780224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:31:52.780224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:31:52.780224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:31:52.780224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:31:52.780224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:31:52.780224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:31:52.780224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 13 20:31:52.780224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 647/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 13 20:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:52.785707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e60f1-ee0a-46cb-a426-fc51c88ca51c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:52.838926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:52.888083 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e60f1-ee0a-46cb-a426-fc51c88ca51c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:52.922403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818687f-151d-46bb-8589-543de4e385f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:52.923061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818687f-151d-46bb-8589-543de4e385f4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:52.923342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818687f-151d-46bb-8589-543de4e385f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:52.923644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818687f-151d-46bb-8589-543de4e385f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:52.924430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818687f-151d-46bb-8589-543de4e385f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:52.924736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818687f-151d-46bb-8589-543de4e385f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:52.926240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818687f-151d-46bb-8589-543de4e385f4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:52.938067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 639/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 13 20:31:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:52.948330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65190b-07a2-4345-ad1f-2066c66f57fb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:53.052140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65190b-07a2-4345-ad1f-2066c66f57fb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:53.094045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b586988-3816-4a40-91d8-016e1c12fd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:53.094698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b586988-3816-4a40-91d8-016e1c12fd4f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:53.094977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b586988-3816-4a40-91d8-016e1c12fd4f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:53.095251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b586988-3816-4a40-91d8-016e1c12fd4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:53.096112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b586988-3816-4a40-91d8-016e1c12fd4f None admin] RBAC: Adding query filter params (role.id=d8b662bc41c14d36a5b5a94ff86d61ef, scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06, user.id=cbafea9ed6494a3f9aeba4140a32de4f) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:53.096420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b586988-3816-4a40-91d8-016e1c12fd4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:53.097850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b586988-3816-4a40-91d8-016e1c12fd4f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:53.103914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 648/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 13 20:31:52 2022] GET /identity/v3/role_assignments?user.id=cbafea9ed6494a3f9aeba4140a32de4f&scope.project.id=ef928cdf3dcd4d7392f3749bf55bec06&role.id=d8b662bc41c14d36a5b5a94ff86d61ef => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:55.149328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82304a0-1a91-4fe1-90a0-75eefa9480bf None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:55.150245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82304a0-1a91-4fe1-90a0-75eefa9480bf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:55.150245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82304a0-1a91-4fe1-90a0-75eefa9480bf None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:55.152434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 640/1288] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:31:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:31:55.161194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2316-cb7f-4507-ab95-ce45cb572afe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:55.161635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2316-cb7f-4507-ab95-ce45cb572afe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:55.161802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2316-cb7f-4507-ab95-ce45cb572afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:55.184271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-928c2316-cb7f-4507-ab95-ce45cb572afe None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:31:55.185472 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-928c2316-cb7f-4507-ab95-ce45cb572afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:31:55.217601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:55.258872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 649/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:31:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:31:55.282049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82304a0-1a91-4fe1-90a0-75eefa9480bf None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:55.386607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82304a0-1a91-4fe1-90a0-75eefa9480bf None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:55.397138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:55.425447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15ed83-ac73-40d2-8deb-b0a9c98c454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:55.426243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15ed83-ac73-40d2-8deb-b0a9c98c454b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:55.426529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15ed83-ac73-40d2-8deb-b0a9c98c454b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:55.426783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15ed83-ac73-40d2-8deb-b0a9c98c454b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:55.427427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15ed83-ac73-40d2-8deb-b0a9c98c454b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:55.428859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15ed83-ac73-40d2-8deb-b0a9c98c454b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1d15ed83-ac73-40d2-8deb-b0a9c98c454b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:31:55.440876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:31:55.442020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:31:55.443388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 641/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:31:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:31:55.450661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2316-cb7f-4507-ab95-ce45cb572afe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:55.556209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2316-cb7f-4507-ab95-ce45cb572afe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:55.595881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb109284-f01e-4b57-a0d6-1e283face03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:55.596535 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb109284-f01e-4b57-a0d6-1e283face03a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:55.596862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb109284-f01e-4b57-a0d6-1e283face03a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:55.597133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb109284-f01e-4b57-a0d6-1e283face03a None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:55.597839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb109284-f01e-4b57-a0d6-1e283face03a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:55.598147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb109284-f01e-4b57-a0d6-1e283face03a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:55.599500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb109284-f01e-4b57-a0d6-1e283face03a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:55.604641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 650/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Jul 13 20:31:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:55.611280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15ed83-ac73-40d2-8deb-b0a9c98c454b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:55.719364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15ed83-ac73-40d2-8deb-b0a9c98c454b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:55.754008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075738a-74b9-4754-8eb6-2eb59d30eed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:55.754651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075738a-74b9-4754-8eb6-2eb59d30eed3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:55.754939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075738a-74b9-4754-8eb6-2eb59d30eed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:55.755214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075738a-74b9-4754-8eb6-2eb59d30eed3 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:55.755914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075738a-74b9-4754-8eb6-2eb59d30eed3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:31:55.756221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075738a-74b9-4754-8eb6-2eb59d30eed3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:55.757595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075738a-74b9-4754-8eb6-2eb59d30eed3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:55.762221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 642/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Jul 13 20:31:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:31:57.752108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569767e1-f390-47ac-a484-d59033db852a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:57.752961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569767e1-f390-47ac-a484-d59033db852a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:57.753112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569767e1-f390-47ac-a484-d59033db852a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:57.757906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 651/1293] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:31:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:31:57.763944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ce255-1255-43cb-a5c9-4e3235e5622c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:57.764372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ce255-1255-43cb-a5c9-4e3235e5622c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:57.764549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ce255-1255-43cb-a5c9-4e3235e5622c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:57.790870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-301ce255-1255-43cb-a5c9-4e3235e5622c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:31:57.792237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-301ce255-1255-43cb-a5c9-4e3235e5622c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:31:57.829181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:57.879488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 643/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:31:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:31:57.900166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569767e1-f390-47ac-a484-d59033db852a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:31:57.935065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:31:58.012402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569767e1-f390-47ac-a484-d59033db852a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:31:58.047151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4d73f-c687-4bdf-b1d2-98920c4ea036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:31:58.047832 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4d73f-c687-4bdf-b1d2-98920c4ea036 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:31:58.048121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4d73f-c687-4bdf-b1d2-98920c4ea036 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:31:58.048395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4d73f-c687-4bdf-b1d2-98920c4ea036 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:31:58.049104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4d73f-c687-4bdf-b1d2-98920c4ea036 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:31:58.050507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4d73f-c687-4bdf-b1d2-98920c4ea036 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:31:58.068247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 652/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 13 20:31:57 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:32:00.152437 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad86c-4780-4de1-b037-3fc56e4836b6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:00.152979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad86c-4780-4de1-b037-3fc56e4836b6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:00.152979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad86c-4780-4de1-b037-3fc56e4836b6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:00.154394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 644/1296] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:32:00.160736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ff307-f894-4f85-a3db-90567a370457 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:00.161447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ff307-f894-4f85-a3db-90567a370457 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:00.161601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ff307-f894-4f85-a3db-90567a370457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:00.190731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7a4ff307-f894-4f85-a3db-90567a370457 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:32:00.191983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7a4ff307-f894-4f85-a3db-90567a370457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:32:00.269745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 653/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:32:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:32:00.297982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebad86c-4780-4de1-b037-3fc56e4836b6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:32:00.406439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebad86c-4780-4de1-b037-3fc56e4836b6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:32:00.442477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318617bf-fa9f-4ea4-b40b-d63161843e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:32:00.443272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318617bf-fa9f-4ea4-b40b-d63161843e64 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:00.443593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318617bf-fa9f-4ea4-b40b-d63161843e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:00.444164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318617bf-fa9f-4ea4-b40b-d63161843e64 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:00.444850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318617bf-fa9f-4ea4-b40b-d63161843e64 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:32:00.446244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318617bf-fa9f-4ea4-b40b-d63161843e64 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-318617bf-fa9f-4ea4-b40b-d63161843e64 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:32:00.461241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:32:00.462340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:32:00.463307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 645/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:32:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:32:00.468753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ff307-f894-4f85-a3db-90567a370457 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:32:00.488657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:32:00.580259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ff307-f894-4f85-a3db-90567a370457 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:32:00.614870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76cd86-b7bd-4b5e-9243-d2975c0e2a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:32:00.615550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76cd86-b7bd-4b5e-9243-d2975c0e2a59 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:00.615838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76cd86-b7bd-4b5e-9243-d2975c0e2a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:00.616120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76cd86-b7bd-4b5e-9243-d2975c0e2a59 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:00.616831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76cd86-b7bd-4b5e-9243-d2975c0e2a59 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:32:00.617143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76cd86-b7bd-4b5e-9243-d2975c0e2a59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:32:00.618540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76cd86-b7bd-4b5e-9243-d2975c0e2a59 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:32:00.624199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 654/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Jul 13 20:32:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:32:00.630846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318617bf-fa9f-4ea4-b40b-d63161843e64 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:32:00.669563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:32:00.744080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318617bf-fa9f-4ea4-b40b-d63161843e64 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:32:00.781694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1200d-bff6-4090-bbab-755681a90a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:32:00.782342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1200d-bff6-4090-bbab-755681a90a5d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:00.782663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1200d-bff6-4090-bbab-755681a90a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:00.782942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1200d-bff6-4090-bbab-755681a90a5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:00.783767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1200d-bff6-4090-bbab-755681a90a5d None admin] RBAC: Adding query filter params (interface=public, service_id=2757cb44c0e14eb2935cb5457846b4e0, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:32:00.784150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1200d-bff6-4090-bbab-755681a90a5d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:32:00.785553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1200d-bff6-4090-bbab-755681a90a5d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:32:00.790707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 646/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 13 20:32:00 2022] GET /identity/v3/endpoints?service_id=2757cb44c0e14eb2935cb5457846b4e0&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:32:00.796610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76cd86-b7bd-4b5e-9243-d2975c0e2a59 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:32:00.881986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:32:00.902568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76cd86-b7bd-4b5e-9243-d2975c0e2a59 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:32:00.938223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5e265-be22-435b-a6e5-d0ddf5a7640a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:32:00.938814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e265-be22-435b-a6e5-d0ddf5a7640a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:00.939107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e265-be22-435b-a6e5-d0ddf5a7640a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:00.939386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e265-be22-435b-a6e5-d0ddf5a7640a None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:00.940106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5e265-be22-435b-a6e5-d0ddf5a7640a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:32:00.941556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5e265-be22-435b-a6e5-d0ddf5a7640a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:32:00.948798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 655/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 13 20:32:00 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:32:02.937854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf60065-ab3d-4247-af62-e7cdf7c7de3e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:02.938696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf60065-ab3d-4247-af62-e7cdf7c7de3e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:02.938696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf60065-ab3d-4247-af62-e7cdf7c7de3e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:02.941005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 647/1302] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:32:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:32:02.949998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ed4c7-4677-4bd4-a8d6-0313ef792f62 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:02.950402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ed4c7-4677-4bd4-a8d6-0313ef792f62 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:02.950602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ed4c7-4677-4bd4-a8d6-0313ef792f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:02.970537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-de1ed4c7-4677-4bd4-a8d6-0313ef792f62 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:32:02.971842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-de1ed4c7-4677-4bd4-a8d6-0313ef792f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:32:03.039987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 656/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:32:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:32:03.062542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf60065-ab3d-4247-af62-e7cdf7c7de3e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:32:03.171151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:32:03.181806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf60065-ab3d-4247-af62-e7cdf7c7de3e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:32:03.221593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bf690-e3c4-4a9e-9d36-2879a7ee4eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:32:03.222389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bf690-e3c4-4a9e-9d36-2879a7ee4eaf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:03.222674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bf690-e3c4-4a9e-9d36-2879a7ee4eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:03.222959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bf690-e3c4-4a9e-9d36-2879a7ee4eaf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:03.223686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bf690-e3c4-4a9e-9d36-2879a7ee4eaf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:32:03.225073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bf690-e3c4-4a9e-9d36-2879a7ee4eaf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2a6bf690-e3c4-4a9e-9d36-2879a7ee4eaf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:32:03.237432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:32:03.238567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 13 20:32:03.239623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 648/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 13 20:32:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:32:03.245156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ed4c7-4677-4bd4-a8d6-0313ef792f62 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:32:03.323967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:32:03.351183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ed4c7-4677-4bd4-a8d6-0313ef792f62 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:32:03.388007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda32d7a-55cc-4099-93cd-70b6b17dde08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:32:03.388667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda32d7a-55cc-4099-93cd-70b6b17dde08 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:03.388956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda32d7a-55cc-4099-93cd-70b6b17dde08 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:03.389270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda32d7a-55cc-4099-93cd-70b6b17dde08 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:03.389986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda32d7a-55cc-4099-93cd-70b6b17dde08 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:32:03.390296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda32d7a-55cc-4099-93cd-70b6b17dde08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:32:03.391711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda32d7a-55cc-4099-93cd-70b6b17dde08 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:32:03.397076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 657/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Jul 13 20:32:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:32:03.403191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bf690-e3c4-4a9e-9d36-2879a7ee4eaf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:32:03.504039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bf690-e3c4-4a9e-9d36-2879a7ee4eaf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:32:03.538535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4691b3d-e520-418a-8986-3f53fe8cc22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:32:03.539174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4691b3d-e520-418a-8986-3f53fe8cc22b None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:32:03.539493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4691b3d-e520-418a-8986-3f53fe8cc22b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:32:03.539773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4691b3d-e520-418a-8986-3f53fe8cc22b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:32:03.540503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4691b3d-e520-418a-8986-3f53fe8cc22b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:32:03.541890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4691b3d-e520-418a-8986-3f53fe8cc22b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:32:03.563186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 649/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 13 20:32:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:32:13.394059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=30 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:32:13.559470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:06.704100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcdf97-d052-4cb1-96d2-c41f28d62e77 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:06.704622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcdf97-d052-4cb1-96d2-c41f28d62e77 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:06.704717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcdf97-d052-4cb1-96d2-c41f28d62e77 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:06.710830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 658/1307] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:06.712056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6e36c-d7cb-433a-8464-29c9e131ad88 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:06.712935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6e36c-d7cb-433a-8464-29c9e131ad88 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:06.714588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6e36c-d7cb-433a-8464-29c9e131ad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:06.791345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-24c6e36c-d7cb-433a-8464-29c9e131ad88 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:06.792595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-24c6e36c-d7cb-433a-8464-29c9e131ad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:06.916238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 650/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:06.962952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98158ba-468d-475e-966e-659cf1857f44 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:06.963255 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98158ba-468d-475e-966e-659cf1857f44 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:06.963540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98158ba-468d-475e-966e-659cf1857f44 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:06.965227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 659/1309] 10.222.0.47 () {54 vars in 964 bytes} [Wed Jul 13 20:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:06.975367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3e6e9-6c79-4798-9ca4-32a76b80537a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:06.975713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3e6e9-6c79-4798-9ca4-32a76b80537a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:06.975948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3e6e9-6c79-4798-9ca4-32a76b80537a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:07.062249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-21c3e6e9-6c79-4798-9ca4-32a76b80537a None None] MFA Rules not processed for user `c57d44c19d15496e96c6e37510ce67fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:07.063771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-21c3e6e9-6c79-4798-9ca4-32a76b80537a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:07.099741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:07.172318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 651/1310] 10.222.0.47 () {58 vars in 1061 bytes} [Wed Jul 13 20:33:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:07.181808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98158ba-468d-475e-966e-659cf1857f44 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:07.183927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d98158ba-468d-475e-966e-659cf1857f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:07.302464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98158ba-468d-475e-966e-659cf1857f44 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:07.382382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aaa82-ceb3-446c-9831-9d4be8bcf0c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:07.383500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aaa82-ceb3-446c-9831-9d4be8bcf0c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:07.384040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aaa82-ceb3-446c-9831-9d4be8bcf0c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:07.384536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aaa82-ceb3-446c-9831-9d4be8bcf0c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:07.386612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-307aaa82-ceb3-446c-9831-9d4be8bcf0c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:07.448443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:07.504939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307aaa82-ceb3-446c-9831-9d4be8bcf0c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:07.506384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307aaa82-ceb3-446c-9831-9d4be8bcf0c4 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:07.525207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 660/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:09.825610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14c5a0-626c-4cfd-a5b3-14a42c8ccbc4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:09.826151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14c5a0-626c-4cfd-a5b3-14a42c8ccbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:09.826151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14c5a0-626c-4cfd-a5b3-14a42c8ccbc4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:09.827624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 652/1312] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:09.832817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f0369-91a7-454f-8f90-fe3124705ea1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:09.833272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f0369-91a7-454f-8f90-fe3124705ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:09.833354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f0369-91a7-454f-8f90-fe3124705ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:09.923338 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-843f0369-91a7-454f-8f90-fe3124705ea1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:09.924710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-843f0369-91a7-454f-8f90-fe3124705ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:10.028379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 661/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:10.078105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b3cfd-cbcd-4da6-883d-e0a229f4fe23 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:10.078295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b3cfd-cbcd-4da6-883d-e0a229f4fe23 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:10.078677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b3cfd-cbcd-4da6-883d-e0a229f4fe23 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:10.080344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 653/1314] 10.222.0.47 () {54 vars in 964 bytes} [Wed Jul 13 20:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:10.086308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530942d2-3d51-45f8-aa9c-53fccd00416b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:10.086582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530942d2-3d51-45f8-aa9c-53fccd00416b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:10.086845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530942d2-3d51-45f8-aa9c-53fccd00416b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:10.098357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:10.172798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-530942d2-3d51-45f8-aa9c-53fccd00416b None None] MFA Rules not processed for user `c57d44c19d15496e96c6e37510ce67fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:10.174023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-530942d2-3d51-45f8-aa9c-53fccd00416b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:10.254436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 662/1315] 10.222.0.47 () {58 vars in 1061 bytes} [Wed Jul 13 20:33:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:10.263353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b3cfd-cbcd-4da6-883d-e0a229f4fe23 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:10.264901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-da0b3cfd-cbcd-4da6-883d-e0a229f4fe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:10.382947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b3cfd-cbcd-4da6-883d-e0a229f4fe23 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:10.464709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:10.467122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83b0c-c22e-4b97-98d5-e5cb54e5f0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:10.467899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83b0c-c22e-4b97-98d5-e5cb54e5f0ab service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:10.468440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83b0c-c22e-4b97-98d5-e5cb54e5f0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:10.468890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83b0c-c22e-4b97-98d5-e5cb54e5f0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:10.470980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5dc83b0c-c22e-4b97-98d5-e5cb54e5f0ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:10.575414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc83b0c-c22e-4b97-98d5-e5cb54e5f0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:10.576925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc83b0c-c22e-4b97-98d5-e5cb54e5f0ab service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:10.597294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 654/1316] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:13.157710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e9b30-ba8c-4986-bc60-589168c822cd None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:13.158250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e9b30-ba8c-4986-bc60-589168c822cd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:13.158250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e9b30-ba8c-4986-bc60-589168c822cd None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:13.159672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 663/1317] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:13.165983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57436f5-a430-450b-a916-10398b089e57 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:13.166786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57436f5-a430-450b-a916-10398b089e57 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:13.166912 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57436f5-a430-450b-a916-10398b089e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:13.235293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c57436f5-a430-450b-a916-10398b089e57 None None] MFA Rules not processed for user `6d22fba8da874faab7222240ee9e9e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:13.236525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c57436f5-a430-450b-a916-10398b089e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:13.289231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:13.335040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 655/1318] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:13.383007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e9b30-ba8c-4986-bc60-589168c822cd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:13.384576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-753e9b30-ba8c-4986-bc60-589168c822cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:13.496123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e9b30-ba8c-4986-bc60-589168c822cd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:13.499527 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:13.572403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72456be-b69d-4e9c-96bd-667356bc42ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:13.573059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72456be-b69d-4e9c-96bd-667356bc42ae service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:13.573343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72456be-b69d-4e9c-96bd-667356bc42ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:13.573617 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72456be-b69d-4e9c-96bd-667356bc42ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:13.575245 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f72456be-b69d-4e9c-96bd-667356bc42ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:13.684241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72456be-b69d-4e9c-96bd-667356bc42ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:13.685630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72456be-b69d-4e9c-96bd-667356bc42ae service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:13.705583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 664/1319] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:16.428418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109e05b-1986-416c-a883-90e9791bffaa None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:16.429227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109e05b-1986-416c-a883-90e9791bffaa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:16.429227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109e05b-1986-416c-a883-90e9791bffaa None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:16.431296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 656/1320] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:16.436643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491b045-e7e7-4ab6-a722-e5b63725e7bd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:16.437095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491b045-e7e7-4ab6-a722-e5b63725e7bd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:16.437214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491b045-e7e7-4ab6-a722-e5b63725e7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:16.479170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:16.506711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c491b045-e7e7-4ab6-a722-e5b63725e7bd None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:16.507981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c491b045-e7e7-4ab6-a722-e5b63725e7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:16.613740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 665/1321] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:16.656001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109e05b-1986-416c-a883-90e9791bffaa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:16.657561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1109e05b-1986-416c-a883-90e9791bffaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:16.774593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109e05b-1986-416c-a883-90e9791bffaa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:16.851875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4420c3d-be2a-49bd-95a9-30fad84f60b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:16.852551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4420c3d-be2a-49bd-95a9-30fad84f60b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:16.852813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4420c3d-be2a-49bd-95a9-30fad84f60b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:16.853090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4420c3d-be2a-49bd-95a9-30fad84f60b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:16.854733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a4420c3d-be2a-49bd-95a9-30fad84f60b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:16.862277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:16.968639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4420c3d-be2a-49bd-95a9-30fad84f60b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:16.970102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4420c3d-be2a-49bd-95a9-30fad84f60b8 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:16.989438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 657/1322] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:19.694307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c5a0-1272-4970-918e-2cadbaad77c2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:19.694931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c5a0-1272-4970-918e-2cadbaad77c2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:19.694931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c5a0-1272-4970-918e-2cadbaad77c2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:19.696288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 666/1323] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:19.701619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ed351-10e0-40f2-ac44-8dec77163980 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:19.702060 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ed351-10e0-40f2-ac44-8dec77163980 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:19.702252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ed351-10e0-40f2-ac44-8dec77163980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:19.770703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c87ed351-10e0-40f2-ac44-8dec77163980 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:19.771919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c87ed351-10e0-40f2-ac44-8dec77163980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:19.832201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:19.875322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 658/1324] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:19.911518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c5a0-1272-4970-918e-2cadbaad77c2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:19.913058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1690c5a0-1272-4970-918e-2cadbaad77c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:20.031919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c5a0-1272-4970-918e-2cadbaad77c2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:20.046303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:20.112620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5bba2-4a5e-406a-8695-b1f950b380d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:20.113277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5bba2-4a5e-406a-8695-b1f950b380d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:20.113567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5bba2-4a5e-406a-8695-b1f950b380d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:20.113846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5bba2-4a5e-406a-8695-b1f950b380d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:20.115537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-05c5bba2-4a5e-406a-8695-b1f950b380d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:20.232334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5bba2-4a5e-406a-8695-b1f950b380d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:20.233809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5bba2-4a5e-406a-8695-b1f950b380d9 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:20.252821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 667/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:22.307514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a26f0-1219-4150-afe0-1692158509df None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:22.308055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a26f0-1219-4150-afe0-1692158509df None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:22.308055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a26f0-1219-4150-afe0-1692158509df None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:22.309325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 659/1326] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:22.314733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076254a7-2324-4708-8afd-17b411bb9577 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:22.315160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076254a7-2324-4708-8afd-17b411bb9577 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:22.315325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076254a7-2324-4708-8afd-17b411bb9577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:22.362682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:22.381270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-076254a7-2324-4708-8afd-17b411bb9577 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:22.382475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-076254a7-2324-4708-8afd-17b411bb9577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:22.486290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 668/1327] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:22.521844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a26f0-1219-4150-afe0-1692158509df None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:22.523421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-048a26f0-1219-4150-afe0-1692158509df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:22.633485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a26f0-1219-4150-afe0-1692158509df None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:22.712505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c984ff-d6ca-4077-96dc-d8d1032b8e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:22.713172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c984ff-d6ca-4077-96dc-d8d1032b8e79 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:22.713448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c984ff-d6ca-4077-96dc-d8d1032b8e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:22.713722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c984ff-d6ca-4077-96dc-d8d1032b8e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:22.715437 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-39c984ff-d6ca-4077-96dc-d8d1032b8e79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:22.733036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:22.831879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c984ff-d6ca-4077-96dc-d8d1032b8e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:22.833343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c984ff-d6ca-4077-96dc-d8d1032b8e79 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:22.858153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 660/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:24.949358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6b318-cd81-4e0c-9e69-a90b75f4ef0f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:24.949894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6b318-cd81-4e0c-9e69-a90b75f4ef0f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:24.949894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6b318-cd81-4e0c-9e69-a90b75f4ef0f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:24.951274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 669/1329] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:24.956653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ce653-0731-4cb1-89e6-7e602948c759 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:24.957118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ce653-0731-4cb1-89e6-7e602948c759 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:24.957223 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ce653-0731-4cb1-89e6-7e602948c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:25.025846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c13ce653-0731-4cb1-89e6-7e602948c759 None None] MFA Rules not processed for user `6d22fba8da874faab7222240ee9e9e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:25.027058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c13ce653-0731-4cb1-89e6-7e602948c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:25.103879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:25.138213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 661/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:25.175705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6b318-cd81-4e0c-9e69-a90b75f4ef0f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:25.177233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fcf6b318-cd81-4e0c-9e69-a90b75f4ef0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:25.284794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6b318-cd81-4e0c-9e69-a90b75f4ef0f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:25.307717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:25.366263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8632a-7f67-457f-bb00-2a7ef77d0c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:25.366955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8632a-7f67-457f-bb00-2a7ef77d0c76 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:25.367241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8632a-7f67-457f-bb00-2a7ef77d0c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:25.367521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8632a-7f67-457f-bb00-2a7ef77d0c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:25.369181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0eb8632a-7f67-457f-bb00-2a7ef77d0c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:25.491092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8632a-7f67-457f-bb00-2a7ef77d0c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:25.492567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8632a-7f67-457f-bb00-2a7ef77d0c76 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:25.516057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 670/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:27.789520 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a248-a0ae-46b5-8973-4c5829843fb5 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:27.790352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a248-a0ae-46b5-8973-4c5829843fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:27.790452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a248-a0ae-46b5-8973-4c5829843fb5 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:27.792084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 662/1332] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:27.801440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47849b-c4b6-48c1-bf2b-440991bfdb04 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:27.801927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47849b-c4b6-48c1-bf2b-440991bfdb04 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:27.802100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47849b-c4b6-48c1-bf2b-440991bfdb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:27.867994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:27.872161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4a47849b-c4b6-48c1-bf2b-440991bfdb04 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:27.874678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4a47849b-c4b6-48c1-bf2b-440991bfdb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:27.974861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 671/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:33:28.011672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180a248-a0ae-46b5-8973-4c5829843fb5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:28.013257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f180a248-a0ae-46b5-8973-4c5829843fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:28.124586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180a248-a0ae-46b5-8973-4c5829843fb5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:28.197024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b7e84-d2c5-4cce-a362-69057dfd4a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:28.197678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7e84-d2c5-4cce-a362-69057dfd4a5d service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:28.197961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7e84-d2c5-4cce-a362-69057dfd4a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:28.198213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7e84-d2c5-4cce-a362-69057dfd4a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:28.199923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-058b7e84-d2c5-4cce-a362-69057dfd4a5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:28.227123 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:28.312661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b7e84-d2c5-4cce-a362-69057dfd4a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:28.314113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b7e84-d2c5-4cce-a362-69057dfd4a5d service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:28.333438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 663/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:30.385939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a828c-329f-40a9-9aed-c2b0cd322109 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:30.386771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a828c-329f-40a9-9aed-c2b0cd322109 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:30.386771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a828c-329f-40a9-9aed-c2b0cd322109 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:30.389023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 672/1335] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:30.395239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fe9ef-0b2c-42be-ad5c-d1ccef83fe8d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:30.395703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fe9ef-0b2c-42be-ad5c-d1ccef83fe8d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:30.395849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fe9ef-0b2c-42be-ad5c-d1ccef83fe8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:30.467252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-845fe9ef-0b2c-42be-ad5c-d1ccef83fe8d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:30.468526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-845fe9ef-0b2c-42be-ad5c-d1ccef83fe8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:30.550397 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:30.575704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 664/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:30.619393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a828c-329f-40a9-9aed-c2b0cd322109 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:30.621864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-827a828c-329f-40a9-9aed-c2b0cd322109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:30.759000 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a828c-329f-40a9-9aed-c2b0cd322109 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:30.797279 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:30.846956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9818e-c8ef-4616-a2c7-fabcf54468dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:30.847658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9818e-c8ef-4616-a2c7-fabcf54468dc service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:30.847947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9818e-c8ef-4616-a2c7-fabcf54468dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:30.848227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9818e-c8ef-4616-a2c7-fabcf54468dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:30.849891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ffb9818e-c8ef-4616-a2c7-fabcf54468dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:30.965501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9818e-c8ef-4616-a2c7-fabcf54468dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:30.966942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9818e-c8ef-4616-a2c7-fabcf54468dc service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:30.987332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 673/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:33.331954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e349-36b1-4dc2-bacd-29247273a077 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:33.332478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e349-36b1-4dc2-bacd-29247273a077 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:33.332478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e349-36b1-4dc2-bacd-29247273a077 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:33.334531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 665/1338] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:33.342653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7afaf-d823-4026-ab6b-b9779d14d6f6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:33.343135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7afaf-d823-4026-ab6b-b9779d14d6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:33.343366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7afaf-d823-4026-ab6b-b9779d14d6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:33.410462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ada7afaf-d823-4026-ab6b-b9779d14d6f6 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:33.411701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ada7afaf-d823-4026-ab6b-b9779d14d6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:33.420834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:33.513663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 674/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:33:33.549417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805e349-36b1-4dc2-bacd-29247273a077 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:33.550960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7805e349-36b1-4dc2-bacd-29247273a077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:33.659544 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805e349-36b1-4dc2-bacd-29247273a077 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:33.735334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef684a55-7716-482c-97a4-8089cafa7ed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:33.736051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef684a55-7716-482c-97a4-8089cafa7ed6 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:33.736337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef684a55-7716-482c-97a4-8089cafa7ed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:33.736617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef684a55-7716-482c-97a4-8089cafa7ed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:33.738242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ef684a55-7716-482c-97a4-8089cafa7ed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:33.774283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:33.854515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef684a55-7716-482c-97a4-8089cafa7ed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:33.856018 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef684a55-7716-482c-97a4-8089cafa7ed6 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:33.874993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 666/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:37.854720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f08e17-ed55-450f-93a5-352fbdbf6b10 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:37.855240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f08e17-ed55-450f-93a5-352fbdbf6b10 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:37.855240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f08e17-ed55-450f-93a5-352fbdbf6b10 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:37.856593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 675/1341] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:37.862165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596e4c8-5ec5-458a-a26d-f0c6d62aef11 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:37.862600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596e4c8-5ec5-458a-a26d-f0c6d62aef11 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:37.862733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596e4c8-5ec5-458a-a26d-f0c6d62aef11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:37.929511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c596e4c8-5ec5-458a-a26d-f0c6d62aef11 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:37.930739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c596e4c8-5ec5-458a-a26d-f0c6d62aef11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:38.017301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:38.030170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 667/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:38.069101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f08e17-ed55-450f-93a5-352fbdbf6b10 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:38.070710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-83f08e17-ed55-450f-93a5-352fbdbf6b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:38.178875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f08e17-ed55-450f-93a5-352fbdbf6b10 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:38.219244 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:38.256465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a34f78-d55a-4261-ab02-193c9f5934bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:38.257122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a34f78-d55a-4261-ab02-193c9f5934bd service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:38.257406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a34f78-d55a-4261-ab02-193c9f5934bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:38.257685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a34f78-d55a-4261-ab02-193c9f5934bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:38.259299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-94a34f78-d55a-4261-ab02-193c9f5934bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:38.376813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a34f78-d55a-4261-ab02-193c9f5934bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:38.378306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a34f78-d55a-4261-ab02-193c9f5934bd service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:38.401448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 676/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:41.218730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a88ae-811c-4b14-a76b-9567e8f6c6a2 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:41.219321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a88ae-811c-4b14-a76b-9567e8f6c6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:41.219321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a88ae-811c-4b14-a76b-9567e8f6c6a2 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:41.220844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 668/1344] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:41.226445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356d0ee-4b33-4465-823b-2fcdfae6e359 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:41.226869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356d0ee-4b33-4465-823b-2fcdfae6e359 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:41.227011 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356d0ee-4b33-4465-823b-2fcdfae6e359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:41.293178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5356d0ee-4b33-4465-823b-2fcdfae6e359 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:41.294387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5356d0ee-4b33-4465-823b-2fcdfae6e359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:41.313360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:41.395136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 677/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:41.432422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a88ae-811c-4b14-a76b-9567e8f6c6a2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:41.434043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8c5a88ae-811c-4b14-a76b-9567e8f6c6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:41.540822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a88ae-811c-4b14-a76b-9567e8f6c6a2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:41.621072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d687408-37fd-440e-8395-2cfb9d468871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:41.621724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687408-37fd-440e-8395-2cfb9d468871 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:41.622003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687408-37fd-440e-8395-2cfb9d468871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:41.622275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687408-37fd-440e-8395-2cfb9d468871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:41.623946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4d687408-37fd-440e-8395-2cfb9d468871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:41.667142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:41.740448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d687408-37fd-440e-8395-2cfb9d468871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:41.742312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d687408-37fd-440e-8395-2cfb9d468871 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:41.761832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 669/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:46.011877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda653d4-92d9-4154-9e5d-7f6ee63e3317 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:46.012412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda653d4-92d9-4154-9e5d-7f6ee63e3317 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:46.012412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda653d4-92d9-4154-9e5d-7f6ee63e3317 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:46.013823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 678/1347] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:46.019328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba45c5-de2f-480d-bcf6-1d4e2fd9c8d8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:46.019869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba45c5-de2f-480d-bcf6-1d4e2fd9c8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:46.020016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba45c5-de2f-480d-bcf6-1d4e2fd9c8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:46.091792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9dba45c5-de2f-480d-bcf6-1d4e2fd9c8d8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:46.093022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9dba45c5-de2f-480d-bcf6-1d4e2fd9c8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:46.194045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:46.196714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 670/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:46.235025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda653d4-92d9-4154-9e5d-7f6ee63e3317 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:33:46.236594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cda653d4-92d9-4154-9e5d-7f6ee63e3317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:46.352007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda653d4-92d9-4154-9e5d-7f6ee63e3317 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:33:46.400678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:46.429959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13023d55-9caa-4843-8bf4-44f5f2d2c4a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:33:46.430604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13023d55-9caa-4843-8bf4-44f5f2d2c4a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:46.430889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13023d55-9caa-4843-8bf4-44f5f2d2c4a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:46.431164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13023d55-9caa-4843-8bf4-44f5f2d2c4a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:46.432830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-13023d55-9caa-4843-8bf4-44f5f2d2c4a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:46.549325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13023d55-9caa-4843-8bf4-44f5f2d2c4a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:33:46.551383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13023d55-9caa-4843-8bf4-44f5f2d2c4a2 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:33:46.575900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 679/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:33:51.489248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dd11d-8d43-481c-8ca6-114b0c8405b7 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:51.489813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dd11d-8d43-481c-8ca6-114b0c8405b7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:51.490463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dd11d-8d43-481c-8ca6-114b0c8405b7 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:51.492183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 671/1350] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:51.497352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f917b-4c80-4391-b1b2-6f8a2c6ac194 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:51.497790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f917b-4c80-4391-b1b2-6f8a2c6ac194 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:51.498123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f917b-4c80-4391-b1b2-6f8a2c6ac194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:51.583619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ea6f917b-4c80-4391-b1b2-6f8a2c6ac194 None None] MFA Rules not processed for user `cbafea9ed6494a3f9aeba4140a32de4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:51.584837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ea6f917b-4c80-4391-b1b2-6f8a2c6ac194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:51.605712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:51.700610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 680/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:33:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:55.616490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41c2aa-795d-4ea9-8c01-25c34bb2cc3c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:55.616996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41c2aa-795d-4ea9-8c01-25c34bb2cc3c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:55.617243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41c2aa-795d-4ea9-8c01-25c34bb2cc3c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:55.618839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 672/1352] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:55.624574 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58904505-e520-451e-a3d4-564edcde71a6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:55.625397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58904505-e520-451e-a3d4-564edcde71a6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:55.625974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58904505-e520-451e-a3d4-564edcde71a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:55.706485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-58904505-e520-451e-a3d4-564edcde71a6 None None] MFA Rules not processed for user `cbafea9ed6494a3f9aeba4140a32de4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:55.707809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-58904505-e520-451e-a3d4-564edcde71a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:55.783187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:33:55.822462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 681/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:33:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:33:57.193730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b96d8-db2b-47c7-9a42-7749b3d40e60 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:57.194289 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b96d8-db2b-47c7-9a42-7749b3d40e60 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:57.194289 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b96d8-db2b-47c7-9a42-7749b3d40e60 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:57.195703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 673/1354] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:33:57.200889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dfa42-598a-4ca4-97a9-14dc6a0a1c0e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:33:57.201268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dfa42-598a-4ca4-97a9-14dc6a0a1c0e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:33:57.201566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dfa42-598a-4ca4-97a9-14dc6a0a1c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:33:57.292087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e48dfa42-598a-4ca4-97a9-14dc6a0a1c0e None None] MFA Rules not processed for user `cbafea9ed6494a3f9aeba4140a32de4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:33:57.293291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e48dfa42-598a-4ca4-97a9-14dc6a0a1c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:33:57.406952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 682/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:33:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:00.833608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715430-d997-476b-9700-8b4cde943fce None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:00.834606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715430-d997-476b-9700-8b4cde943fce None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:00.835111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715430-d997-476b-9700-8b4cde943fce None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:00.837041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 674/1356] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:00.842840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f810fa-3a5a-4239-9c12-66ab4b95949b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:00.843324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f810fa-3a5a-4239-9c12-66ab4b95949b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:00.843420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f810fa-3a5a-4239-9c12-66ab4b95949b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:00.890471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:00.930812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-36f810fa-3a5a-4239-9c12-66ab4b95949b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:00.932178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-36f810fa-3a5a-4239-9c12-66ab4b95949b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:01.037124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 683/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:01.100852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e886dec-c395-4248-b4f1-f93e8572292d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:01.101468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e886dec-c395-4248-b4f1-f93e8572292d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:01.101978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e886dec-c395-4248-b4f1-f93e8572292d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:01.104199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 675/1358] 10.222.0.47 () {54 vars in 961 bytes} [Wed Jul 13 20:34:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:01.113667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6019-25c1-40e7-9045-e1a03e61c03e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:01.113972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6019-25c1-40e7-9045-e1a03e61c03e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:01.114235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6019-25c1-40e7-9045-e1a03e61c03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:01.195155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-179e6019-25c1-40e7-9045-e1a03e61c03e None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:01.196388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-179e6019-25c1-40e7-9045-e1a03e61c03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:01.249797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:01.321304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 684/1359] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Jul 13 20:34:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:01.329265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e886dec-c395-4248-b4f1-f93e8572292d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:01.331142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5e886dec-c395-4248-b4f1-f93e8572292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:01.484366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e886dec-c395-4248-b4f1-f93e8572292d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:01.558995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:01.601820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd66d1-6279-42c7-ba39-0996780e899b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:01.602412 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd66d1-6279-42c7-ba39-0996780e899b service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:01.602694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd66d1-6279-42c7-ba39-0996780e899b service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:01.602993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd66d1-6279-42c7-ba39-0996780e899b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:01.604646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4fcd66d1-6279-42c7-ba39-0996780e899b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:01.695627 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd66d1-6279-42c7-ba39-0996780e899b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:01.697262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd66d1-6279-42c7-ba39-0996780e899b service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:01.716266 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 676/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:03.976036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cccb4a-3de6-44ca-afee-a516215c4d5f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:03.976036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cccb4a-3de6-44ca-afee-a516215c4d5f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:03.976558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cccb4a-3de6-44ca-afee-a516215c4d5f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:03.978536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 685/1361] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:34:03.984469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582232a-fba1-4ec3-9e6b-e18dad22c32e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:03.984883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582232a-fba1-4ec3-9e6b-e18dad22c32e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:03.985182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582232a-fba1-4ec3-9e6b-e18dad22c32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:04.063237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9582232a-fba1-4ec3-9e6b-e18dad22c32e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:04.063237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9582232a-fba1-4ec3-9e6b-e18dad22c32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:04.101566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:04.167347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 677/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:04.210653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cccb4a-3de6-44ca-afee-a516215c4d5f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:04.212293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c7cccb4a-3de6-44ca-afee-a516215c4d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:04.362334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cccb4a-3de6-44ca-afee-a516215c4d5f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:04.371815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:04.491728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef56e3-11af-4d9b-b324-fed6441a32aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:04.492255 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef56e3-11af-4d9b-b324-fed6441a32aa service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:04.492790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef56e3-11af-4d9b-b324-fed6441a32aa service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:04.492790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef56e3-11af-4d9b-b324-fed6441a32aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:04.494661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a8ef56e3-11af-4d9b-b324-fed6441a32aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:04.590945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef56e3-11af-4d9b-b324-fed6441a32aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:04.592619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef56e3-11af-4d9b-b324-fed6441a32aa service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:04.611102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 686/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:06.946248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1ea53-fd2a-4777-97af-373a25e9ec5f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:06.946795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1ea53-fd2a-4777-97af-373a25e9ec5f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:06.946795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1ea53-fd2a-4777-97af-373a25e9ec5f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:06.948136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 678/1364] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:06.953564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd567f7-200d-4020-a0d3-586918bc2383 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:06.953980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd567f7-200d-4020-a0d3-586918bc2383 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:06.954136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd567f7-200d-4020-a0d3-586918bc2383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:06.972738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:07.035919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0bd567f7-200d-4020-a0d3-586918bc2383 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:07.037361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0bd567f7-200d-4020-a0d3-586918bc2383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:07.178710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 687/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:07.229227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a416846-b9fb-454e-904d-1074999e5e08 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:07.229418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a416846-b9fb-454e-904d-1074999e5e08 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:07.229828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a416846-b9fb-454e-904d-1074999e5e08 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:07.231495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 679/1366] 10.222.0.47 () {54 vars in 961 bytes} [Wed Jul 13 20:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:07.237679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9defa6f-7016-4784-a37c-189c76370fef None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:07.237975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9defa6f-7016-4784-a37c-189c76370fef None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:07.238260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9defa6f-7016-4784-a37c-189c76370fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:07.323751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c9defa6f-7016-4784-a37c-189c76370fef None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:07.325104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c9defa6f-7016-4784-a37c-189c76370fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:07.361891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:07.449461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 688/1367] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Jul 13 20:34:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:07.458455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a416846-b9fb-454e-904d-1074999e5e08 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:07.460166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2a416846-b9fb-454e-904d-1074999e5e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:07.615128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a416846-b9fb-454e-904d-1074999e5e08 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:07.638190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:07.735246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c1b34-a72c-4b41-b280-c7ea8d84a284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:07.736086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c1b34-a72c-4b41-b280-c7ea8d84a284 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:07.736467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c1b34-a72c-4b41-b280-c7ea8d84a284 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:07.736820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c1b34-a72c-4b41-b280-c7ea8d84a284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:07.738574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6d8c1b34-a72c-4b41-b280-c7ea8d84a284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:07.833280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c1b34-a72c-4b41-b280-c7ea8d84a284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:07.835084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c1b34-a72c-4b41-b280-c7ea8d84a284 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:07.853907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 680/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:08.898495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f9e13-29f8-45be-9aa9-5aaff6119e83 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:08.899047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f9e13-29f8-45be-9aa9-5aaff6119e83 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:08.899047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f9e13-29f8-45be-9aa9-5aaff6119e83 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:08.900579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 689/1369] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:08.906121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554f3e-3762-4de6-8e8a-96b7af19f319 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:08.906557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554f3e-3762-4de6-8e8a-96b7af19f319 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:08.907047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554f3e-3762-4de6-8e8a-96b7af19f319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:08.941383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:08.981731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ef554f3e-3762-4de6-8e8a-96b7af19f319 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:08.983116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ef554f3e-3762-4de6-8e8a-96b7af19f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:09.104417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 681/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:09.459857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bb7e1-80c1-457e-8781-395b4332817c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:09.460212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bb7e1-80c1-457e-8781-395b4332817c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:09.460556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bb7e1-80c1-457e-8781-395b4332817c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:09.462326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 690/1371] 10.222.0.47 () {54 vars in 963 bytes} [Wed Jul 13 20:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:09.467733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720e28-951b-456f-b821-d16a461ff8ad None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:09.468205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720e28-951b-456f-b821-d16a461ff8ad None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:09.468556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720e28-951b-456f-b821-d16a461ff8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:09.549502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-00720e28-951b-456f-b821-d16a461ff8ad None None] MFA Rules not processed for user `9fc23c141eff42f98d31deb2d8aaaca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:09.550699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-00720e28-951b-456f-b821-d16a461ff8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:09.603301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:09.635821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 682/1372] 10.222.0.47 () {58 vars in 1060 bytes} [Wed Jul 13 20:34:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:09.643715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bb7e1-80c1-457e-8781-395b4332817c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:09.645376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b97bb7e1-80c1-457e-8781-395b4332817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:09.756610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bb7e1-80c1-457e-8781-395b4332817c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:09.800608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:09.839400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:09.840085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:09.840351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:09.840629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:09.842250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:09.932552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06fed2-388b-4ec2-b150-3effb50f1dcc None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:09.933084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06fed2-388b-4ec2-b150-3effb50f1dcc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:09.933166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06fed2-388b-4ec2-b150-3effb50f1dcc None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:09.934670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 683/1373] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:34:09.939611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35b5b0-ab51-403b-a3e5-1905c91cc753 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:09.939838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35b5b0-ab51-403b-a3e5-1905c91cc753 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:09.940126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35b5b0-ab51-403b-a3e5-1905c91cc753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:09.955763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:09.957266 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:09.979781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 691/1374] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:10.014718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3a35b5b0-ab51-403b-a3e5-1905c91cc753 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:10.016304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3a35b5b0-ab51-403b-a3e5-1905c91cc753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:10.154796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 684/1375] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:34:10.196769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:10.197978 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:10.270813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:10.309653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea82ce3-24f0-479c-8cbb-9221435fd967 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:10.433534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8392068-be29-446d-8c79-4fc7f9af3728 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:10.434310 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8392068-be29-446d-8c79-4fc7f9af3728 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:10.434593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8392068-be29-446d-8c79-4fc7f9af3728 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:10.434873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8392068-be29-446d-8c79-4fc7f9af3728 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:10.436550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f8392068-be29-446d-8c79-4fc7f9af3728 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:10.519060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:10.530626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8392068-be29-446d-8c79-4fc7f9af3728 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:10.532359 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8392068-be29-446d-8c79-4fc7f9af3728 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:10.550618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 692/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:12.553613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdb325-6269-4a34-b855-442ab43a02d2 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:12.554173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdb325-6269-4a34-b855-442ab43a02d2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:12.554173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdb325-6269-4a34-b855-442ab43a02d2 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:12.555823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 685/1377] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:12.560987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158c09-71fd-448e-b07a-b6986ba8acbb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:12.561389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158c09-71fd-448e-b07a-b6986ba8acbb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:12.561544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158c09-71fd-448e-b07a-b6986ba8acbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:12.631666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-29158c09-71fd-448e-b07a-b6986ba8acbb None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:12.632776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-29158c09-71fd-448e-b07a-b6986ba8acbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:12.733790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 693/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:12.777596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdb325-6269-4a34-b855-442ab43a02d2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:12.779405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9cfdb325-6269-4a34-b855-442ab43a02d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:12.834570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:12.939523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdb325-6269-4a34-b855-442ab43a02d2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:13.064531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb01026-861a-4dc7-83e5-dc2de133ae84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:13.065190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb01026-861a-4dc7-83e5-dc2de133ae84 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:13.065481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb01026-861a-4dc7-83e5-dc2de133ae84 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:13.066109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb01026-861a-4dc7-83e5-dc2de133ae84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:13.067402 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-aeb01026-861a-4dc7-83e5-dc2de133ae84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:13.079900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:13.156108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb01026-861a-4dc7-83e5-dc2de133ae84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:13.157666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb01026-861a-4dc7-83e5-dc2de133ae84 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:13.176984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 686/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:15.243505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5eb97b-76ff-4f4f-a94e-39213189b520 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:15.244049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5eb97b-76ff-4f4f-a94e-39213189b520 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:15.244049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5eb97b-76ff-4f4f-a94e-39213189b520 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:15.245606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 694/1380] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:15.250765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed2a46-e050-402c-9c59-4e13d5c9199a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:15.251227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed2a46-e050-402c-9c59-4e13d5c9199a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:15.251410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed2a46-e050-402c-9c59-4e13d5c9199a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:15.320312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f7ed2a46-e050-402c-9c59-4e13d5c9199a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:15.321495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f7ed2a46-e050-402c-9c59-4e13d5c9199a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:15.411351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:15.426456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 687/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:15.472195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5eb97b-76ff-4f4f-a94e-39213189b520 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:15.474006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dd5eb97b-76ff-4f4f-a94e-39213189b520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:15.546300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:15.634928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5eb97b-76ff-4f4f-a94e-39213189b520 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:15.751965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b51988-99c4-4606-b736-0785b7e8314b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:15.752647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b51988-99c4-4606-b736-0785b7e8314b service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:15.752926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b51988-99c4-4606-b736-0785b7e8314b service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:15.753192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b51988-99c4-4606-b736-0785b7e8314b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:15.754807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-90b51988-99c4-4606-b736-0785b7e8314b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:15.784139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:15.859857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b51988-99c4-4606-b736-0785b7e8314b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:15.861490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b51988-99c4-4606-b736-0785b7e8314b service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:15.880142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 695/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:17.917712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cdd07-e25b-4358-9602-8ffc0bd40a64 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:17.918298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cdd07-e25b-4358-9602-8ffc0bd40a64 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:17.918298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cdd07-e25b-4358-9602-8ffc0bd40a64 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:17.919707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 688/1383] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:17.925406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db316e3-660f-4ff5-b41c-469f57346274 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:17.926364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db316e3-660f-4ff5-b41c-469f57346274 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:17.926364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db316e3-660f-4ff5-b41c-469f57346274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:17.997978 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4db316e3-660f-4ff5-b41c-469f57346274 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:17.999195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4db316e3-660f-4ff5-b41c-469f57346274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:18.096090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:18.099828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 696/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:18.140606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cdd07-e25b-4358-9602-8ffc0bd40a64 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:18.142176 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f48cdd07-e25b-4358-9602-8ffc0bd40a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:18.306638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cdd07-e25b-4358-9602-8ffc0bd40a64 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:18.372853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:18.426482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac5bd-c5be-496b-bbdb-57623aeefebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:18.427136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac5bd-c5be-496b-bbdb-57623aeefebf service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:18.427407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac5bd-c5be-496b-bbdb-57623aeefebf service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:18.427750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac5bd-c5be-496b-bbdb-57623aeefebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:18.429372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4f5ac5bd-c5be-496b-bbdb-57623aeefebf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:18.520754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac5bd-c5be-496b-bbdb-57623aeefebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:18.522428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac5bd-c5be-496b-bbdb-57623aeefebf service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:18.543671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 689/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:20.548995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f5e62-da46-41a9-afd8-379d59a1ad0e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:20.549795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f5e62-da46-41a9-afd8-379d59a1ad0e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:20.549795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f5e62-da46-41a9-afd8-379d59a1ad0e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:20.551967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 697/1386] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:20.559583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adb245-b4f0-40e5-8651-b86f2e812e9c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:20.560053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adb245-b4f0-40e5-8651-b86f2e812e9c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:20.560419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adb245-b4f0-40e5-8651-b86f2e812e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:20.635588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f0adb245-b4f0-40e5-8651-b86f2e812e9c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:20.638156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f0adb245-b4f0-40e5-8651-b86f2e812e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:20.674474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:20.772445 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 690/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:20.828561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f5e62-da46-41a9-afd8-379d59a1ad0e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:20.830159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7b1f5e62-da46-41a9-afd8-379d59a1ad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:20.988841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f5e62-da46-41a9-afd8-379d59a1ad0e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:21.064719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:21.121342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa701b-6f77-41c1-bcec-0b228cf57b88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:21.122003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa701b-6f77-41c1-bcec-0b228cf57b88 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:21.122287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa701b-6f77-41c1-bcec-0b228cf57b88 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:21.122562 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa701b-6f77-41c1-bcec-0b228cf57b88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:21.124315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e1aa701b-6f77-41c1-bcec-0b228cf57b88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:21.221642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa701b-6f77-41c1-bcec-0b228cf57b88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:21.223271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa701b-6f77-41c1-bcec-0b228cf57b88 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:21.241665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 698/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:23.281252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9ade-8686-4660-bf92-2fa8582f0a82 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:23.281948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9ade-8686-4660-bf92-2fa8582f0a82 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:23.281948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9ade-8686-4660-bf92-2fa8582f0a82 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:23.283451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 691/1389] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:23.289162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22e6f3-8c07-46d3-844d-15aca99f2886 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:23.289660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22e6f3-8c07-46d3-844d-15aca99f2886 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:23.289774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22e6f3-8c07-46d3-844d-15aca99f2886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:23.358229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8f22e6f3-8c07-46d3-844d-15aca99f2886 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:23.359355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8f22e6f3-8c07-46d3-844d-15aca99f2886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:23.395672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:23.463643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 699/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:34:23.505901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca9ade-8686-4660-bf92-2fa8582f0a82 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:23.507501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7bca9ade-8686-4660-bf92-2fa8582f0a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:23.657448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca9ade-8686-4660-bf92-2fa8582f0a82 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:23.666597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:23.772024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bb139-323a-4f70-8389-35dac3750d05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:23.772683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bb139-323a-4f70-8389-35dac3750d05 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:23.772961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bb139-323a-4f70-8389-35dac3750d05 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:23.773236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bb139-323a-4f70-8389-35dac3750d05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:23.774856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-aa6bb139-323a-4f70-8389-35dac3750d05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:23.864909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bb139-323a-4f70-8389-35dac3750d05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:23.866540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bb139-323a-4f70-8389-35dac3750d05 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:23.885061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 692/1391] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:24.944334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f3c5d-fc1b-4b92-ab81-90e816773e13 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:24.945196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f3c5d-fc1b-4b92-ab81-90e816773e13 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:24.945379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f3c5d-fc1b-4b92-ab81-90e816773e13 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:24.947005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 700/1392] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:24.952298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15186dca-9b84-483d-bd2f-0d468442f998 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:24.952873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15186dca-9b84-483d-bd2f-0d468442f998 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:24.952873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15186dca-9b84-483d-bd2f-0d468442f998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:24.967106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:25.020983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-15186dca-9b84-483d-bd2f-0d468442f998 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:25.022114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-15186dca-9b84-483d-bd2f-0d468442f998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:25.127693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 693/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:25.182810 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb35a3-5be4-43c7-bf03-52a62bc09f86 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:25.183172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb35a3-5be4-43c7-bf03-52a62bc09f86 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:25.183471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb35a3-5be4-43c7-bf03-52a62bc09f86 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:25.184965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 701/1394] 10.222.0.47 () {54 vars in 962 bytes} [Wed Jul 13 20:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:25.190268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d811a-31ad-4031-9761-a65ac1e4e584 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:25.190563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d811a-31ad-4031-9761-a65ac1e4e584 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:25.190819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d811a-31ad-4031-9761-a65ac1e4e584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:25.266425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-335d811a-31ad-4031-9761-a65ac1e4e584 None None] MFA Rules not processed for user `cc21faf2237d405bba03e5309d93629f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:25.267651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-335d811a-31ad-4031-9761-a65ac1e4e584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:25.301859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:25.357543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 694/1395] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Jul 13 20:34:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:25.365643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb35a3-5be4-43c7-bf03-52a62bc09f86 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:25.367238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7cdb35a3-5be4-43c7-bf03-52a62bc09f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:25.481263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb35a3-5be4-43c7-bf03-52a62bc09f86 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:25.548196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:25.561376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b2b55-b1c3-4d27-9e00-228393eb284a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc21faf2237d405bba03e5309d93629f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:25.562239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b2b55-b1c3-4d27-9e00-228393eb284a service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:25.562628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b2b55-b1c3-4d27-9e00-228393eb284a service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:25.563077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b2b55-b1c3-4d27-9e00-228393eb284a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:25.565504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-428b2b55-b1c3-4d27-9e00-228393eb284a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:25.685202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b2b55-b1c3-4d27-9e00-228393eb284a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:25.686661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b2b55-b1c3-4d27-9e00-228393eb284a service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:25.705712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 702/1396] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:25.932197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf655b24-6dad-43a2-b6fd-0ccfcc84603e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:25.932438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf655b24-6dad-43a2-b6fd-0ccfcc84603e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:25.932721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf655b24-6dad-43a2-b6fd-0ccfcc84603e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:25.934181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 695/1397] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:25.940264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b863c5e-8d99-4c2d-9282-fa9d9333b180 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:25.940548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b863c5e-8d99-4c2d-9282-fa9d9333b180 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:25.940825 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b863c5e-8d99-4c2d-9282-fa9d9333b180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:26.014040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4b863c5e-8d99-4c2d-9282-fa9d9333b180 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:26.015358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4b863c5e-8d99-4c2d-9282-fa9d9333b180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:26.074772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:26.129986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 703/1398] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:26.173001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf655b24-6dad-43a2-b6fd-0ccfcc84603e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:26.174797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bf655b24-6dad-43a2-b6fd-0ccfcc84603e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:26.366927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf655b24-6dad-43a2-b6fd-0ccfcc84603e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:26.421777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:26.425315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c3054-3c6d-408c-b325-4291a9ea2d81 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:26.425688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c3054-3c6d-408c-b325-4291a9ea2d81 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:26.425967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c3054-3c6d-408c-b325-4291a9ea2d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:26.492794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1f0c3054-3c6d-408c-b325-4291a9ea2d81 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:26.494054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1f0c3054-3c6d-408c-b325-4291a9ea2d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:26.498803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:26.499466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:26.499764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:26.500036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:26.501764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:26.584380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 704/1399] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:34:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:26.611038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:26.614028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:26.629142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b2156-b4d8-4303-970a-de4163fe5b59 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:26.629438 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b2156-b4d8-4303-970a-de4163fe5b59 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:26.629721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b2156-b4d8-4303-970a-de4163fe5b59 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:26.631128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 705/1400] 10.222.0.47 () {54 vars in 942 bytes} [Wed Jul 13 20:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:26.637007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69369-25a7-4145-93e0-363fdca75a95 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:26.637322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69369-25a7-4145-93e0-363fdca75a95 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:26.637609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69369-25a7-4145-93e0-363fdca75a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:26.640783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 696/1401] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:26.685360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:26.844225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1df69369-25a7-4145-93e0-363fdca75a95 None None] MFA Rules not processed for user `3f27793f58c74ffb9a7a8e4cdddb8169`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:26.845617 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1df69369-25a7-4145-93e0-363fdca75a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:26.959803 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 706/1402] 10.222.0.47 () {58 vars in 1039 bytes} [Wed Jul 13 20:34:26 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:34:26.969378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:26.970613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:27.018767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:27.066546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07c174-f5f4-4072-8fb7-2d426fe2a630 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:27.205249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181c52b-c7fd-463c-afc5-20c5b3a8323d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:27.205959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181c52b-c7fd-463c-afc5-20c5b3a8323d service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:27.206173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181c52b-c7fd-463c-afc5-20c5b3a8323d service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:27.206452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181c52b-c7fd-463c-afc5-20c5b3a8323d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:27.208264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3181c52b-c7fd-463c-afc5-20c5b3a8323d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:27.295342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:27.299379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181c52b-c7fd-463c-afc5-20c5b3a8323d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:27.301482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181c52b-c7fd-463c-afc5-20c5b3a8323d service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:27.309873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 697/1403] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:34:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:34:28.859198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f670-fa5c-41a3-b544-5e8589aa0072 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:28.859840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f670-fa5c-41a3-b544-5e8589aa0072 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:28.859957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f670-fa5c-41a3-b544-5e8589aa0072 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:28.862794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 707/1404] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:28.869952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f5a29-91d9-4bf0-8ae5-173b5f1571b1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:28.870490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f5a29-91d9-4bf0-8ae5-173b5f1571b1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:28.870490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f5a29-91d9-4bf0-8ae5-173b5f1571b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:28.936085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d88f5a29-91d9-4bf0-8ae5-173b5f1571b1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:28.941278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d88f5a29-91d9-4bf0-8ae5-173b5f1571b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:29.041841 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 698/1405] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:29.085797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f670-fa5c-41a3-b544-5e8589aa0072 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:29.087329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-af25f670-fa5c-41a3-b544-5e8589aa0072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:29.190052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:29.248509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f670-fa5c-41a3-b544-5e8589aa0072 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:29.380429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1ed6a-c145-4324-8aa3-b490524a1691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:29.381088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1ed6a-c145-4324-8aa3-b490524a1691 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:29.381439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1ed6a-c145-4324-8aa3-b490524a1691 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:29.381769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1ed6a-c145-4324-8aa3-b490524a1691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:29.383448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-30c1ed6a-c145-4324-8aa3-b490524a1691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:29.444917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:29.482726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1ed6a-c145-4324-8aa3-b490524a1691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:29.485159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1ed6a-c145-4324-8aa3-b490524a1691 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:29.518884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 708/1406] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:31.562797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccf424-ae89-41f6-abc6-6a4c06169370 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:31.563420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccf424-ae89-41f6-abc6-6a4c06169370 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:31.563420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccf424-ae89-41f6-abc6-6a4c06169370 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:31.564864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 699/1407] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:31.570542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711c1f1-7e0c-4eb1-b181-32601765f394 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:31.570999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711c1f1-7e0c-4eb1-b181-32601765f394 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:31.571102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711c1f1-7e0c-4eb1-b181-32601765f394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:31.645994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0711c1f1-7e0c-4eb1-b181-32601765f394 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:31.647215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0711c1f1-7e0c-4eb1-b181-32601765f394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:31.769228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 709/1408] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:31.818002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccf424-ae89-41f6-abc6-6a4c06169370 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:31.820096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3cccf424-ae89-41f6-abc6-6a4c06169370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:31.906441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:31.985895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccf424-ae89-41f6-abc6-6a4c06169370 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:32.112529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e90649-ec8d-4ddd-9c0b-735955d1fa25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:32.113195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e90649-ec8d-4ddd-9c0b-735955d1fa25 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:32.113487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e90649-ec8d-4ddd-9c0b-735955d1fa25 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:32.113766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e90649-ec8d-4ddd-9c0b-735955d1fa25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:32.115472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-32e90649-ec8d-4ddd-9c0b-735955d1fa25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:32.151225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:32.213506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e90649-ec8d-4ddd-9c0b-735955d1fa25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:32.215198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e90649-ec8d-4ddd-9c0b-735955d1fa25 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:32.236791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 700/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:34:41.766787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=85 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:42.234214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=30 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:49.416819 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599367a-6a72-4fbc-bb01-42e0831be07e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:49.417350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599367a-6a72-4fbc-bb01-42e0831be07e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:49.417350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599367a-6a72-4fbc-bb01-42e0831be07e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:49.418710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 710/1410] 10.222.0.47 () {54 vars in 918 bytes} [Wed Jul 13 20:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:49.424360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0f49-c448-4953-8ebc-1f733a56c4d3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:49.424739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0f49-c448-4953-8ebc-1f733a56c4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:49.424952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0f49-c448-4953-8ebc-1f733a56c4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:49.513506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-791c0f49-c448-4953-8ebc-1f733a56c4d3 None None] MFA Rules not processed for user `cbafea9ed6494a3f9aeba4140a32de4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:49.514800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-791c0f49-c448-4953-8ebc-1f733a56c4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:49.639925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 701/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Jul 13 20:34:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:49.655366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69fa8b-0208-43e9-aadb-dc7c893636fe None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:49.655689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69fa8b-0208-43e9-aadb-dc7c893636fe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:49.655997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69fa8b-0208-43e9-aadb-dc7c893636fe None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:49.657418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 711/1412] 10.222.0.47 () {54 vars in 918 bytes} [Wed Jul 13 20:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:49.662206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd751550-b599-411f-8e06-54f563da929b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:49.662502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd751550-b599-411f-8e06-54f563da929b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:49.662784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd751550-b599-411f-8e06-54f563da929b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:49.737625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:49.740346 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cd751550-b599-411f-8e06-54f563da929b None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:49.742288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cd751550-b599-411f-8e06-54f563da929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:49.872683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 702/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Jul 13 20:34:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:52.360040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190878f-c22c-43f2-a269-9dbe93df60d0 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:52.360605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190878f-c22c-43f2-a269-9dbe93df60d0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:52.360605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190878f-c22c-43f2-a269-9dbe93df60d0 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:52.361970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:52.370622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40d5fc-3060-48d0-a2e8-f0891d806f74 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:52.371339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40d5fc-3060-48d0-a2e8-f0891d806f74 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:52.371559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40d5fc-3060-48d0-a2e8-f0891d806f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:52.447217 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7c40d5fc-3060-48d0-a2e8-f0891d806f74 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:52.448485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7c40d5fc-3060-48d0-a2e8-f0891d806f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:52.504309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:52.548729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:52.591376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190878f-c22c-43f2-a269-9dbe93df60d0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:52.592944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6190878f-c22c-43f2-a269-9dbe93df60d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:52.748585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190878f-c22c-43f2-a269-9dbe93df60d0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:52.824288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:52.868771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de3ae2-a0ec-4632-87fd-1a23a519292e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:52.869429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de3ae2-a0ec-4632-87fd-1a23a519292e service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:52.869710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de3ae2-a0ec-4632-87fd-1a23a519292e service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:52.869988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de3ae2-a0ec-4632-87fd-1a23a519292e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:52.871676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-10de3ae2-a0ec-4632-87fd-1a23a519292e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:52.971959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de3ae2-a0ec-4632-87fd-1a23a519292e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:52.973641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de3ae2-a0ec-4632-87fd-1a23a519292e service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:52.992870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 713/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:53.882191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddcc61-d831-4719-aebb-95e13161fc3a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:53.883101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddcc61-d831-4719-aebb-95e13161fc3a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:53.883101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddcc61-d831-4719-aebb-95e13161fc3a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:53.885417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:34:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:53.891400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fb7e8-7f14-4c15-8cdd-ef7ed2a72ef8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:53.891880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fb7e8-7f14-4c15-8cdd-ef7ed2a72ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:53.891985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fb7e8-7f14-4c15-8cdd-ef7ed2a72ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:53.971637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-289fb7e8-7f14-4c15-8cdd-ef7ed2a72ef8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:53.972859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-289fb7e8-7f14-4c15-8cdd-ef7ed2a72ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:54.002136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:54.082077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:54.237299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddcc61-d831-4719-aebb-95e13161fc3a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:54.239823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a3ddcc61-d831-4719-aebb-95e13161fc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:54.396671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddcc61-d831-4719-aebb-95e13161fc3a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:54.441781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:54.528367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93479d-3194-4bfa-a7cf-f4172e43305e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:54.528941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93479d-3194-4bfa-a7cf-f4172e43305e service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:54.529227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93479d-3194-4bfa-a7cf-f4172e43305e service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:54.529506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93479d-3194-4bfa-a7cf-f4172e43305e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:54.531151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6e93479d-3194-4bfa-a7cf-f4172e43305e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:54.626671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93479d-3194-4bfa-a7cf-f4172e43305e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:54.628352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93479d-3194-4bfa-a7cf-f4172e43305e service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:54.647179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 705/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:55.518183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10ab00-5c65-4898-b37f-adaf290fd01c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:55.518785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10ab00-5c65-4898-b37f-adaf290fd01c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:55.518886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10ab00-5c65-4898-b37f-adaf290fd01c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:55.521040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:34:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:55.531098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57904b44-5c7d-4840-838c-b025fd145999 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:55.531594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57904b44-5c7d-4840-838c-b025fd145999 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:55.531799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57904b44-5c7d-4840-838c-b025fd145999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:55.594991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:55.619499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-57904b44-5c7d-4840-838c-b025fd145999 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:55.620773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-57904b44-5c7d-4840-838c-b025fd145999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:55.720640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:55.871848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10ab00-5c65-4898-b37f-adaf290fd01c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:55.873265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-de10ab00-5c65-4898-b37f-adaf290fd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:56.030373 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10ab00-5c65-4898-b37f-adaf290fd01c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:56.043398 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:56.143768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d797c23-dc6a-4c4c-83d7-13b6a4696be4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:56.144348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d797c23-dc6a-4c4c-83d7-13b6a4696be4 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:56.144629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d797c23-dc6a-4c4c-83d7-13b6a4696be4 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:56.144905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d797c23-dc6a-4c4c-83d7-13b6a4696be4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:56.146539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4d797c23-dc6a-4c4c-83d7-13b6a4696be4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:56.242509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d797c23-dc6a-4c4c-83d7-13b6a4696be4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:56.244162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d797c23-dc6a-4c4c-83d7-13b6a4696be4 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:56.263289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 716/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:57.117250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91099048-1834-4848-8221-44883adaa066 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:57.118000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91099048-1834-4848-8221-44883adaa066 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:57.118118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91099048-1834-4848-8221-44883adaa066 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:57.120374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:34:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:57.125905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26022-ed3b-42db-b733-55e978876c17 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:57.126353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26022-ed3b-42db-b733-55e978876c17 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:57.126458 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26022-ed3b-42db-b733-55e978876c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:57.150792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:57.213513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3dc26022-ed3b-42db-b733-55e978876c17 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:57.214726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3dc26022-ed3b-42db-b733-55e978876c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:57.318031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:57.468203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91099048-1834-4848-8221-44883adaa066 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:57.470499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-91099048-1834-4848-8221-44883adaa066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:57.590752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:57.629405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91099048-1834-4848-8221-44883adaa066 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:57.755882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6853d-5d0c-45db-889b-f92241ed7cac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:57.756545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6853d-5d0c-45db-889b-f92241ed7cac service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:57.756839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6853d-5d0c-45db-889b-f92241ed7cac service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:57.757109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6853d-5d0c-45db-889b-f92241ed7cac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:57.758771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1fb6853d-5d0c-45db-889b-f92241ed7cac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:57.841164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:57.856826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6853d-5d0c-45db-889b-f92241ed7cac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:57.858458 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6853d-5d0c-45db-889b-f92241ed7cac service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:57.882049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 708/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:34:58.745054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd299d6-5a0d-496a-aba6-2c8425606433 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:58.745925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd299d6-5a0d-496a-aba6-2c8425606433 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:58.745925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd299d6-5a0d-496a-aba6-2c8425606433 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:58.747961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:34:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:34:58.753948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cbaf2-9153-47a2-828f-00a931533553 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:58.754389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cbaf2-9153-47a2-828f-00a931533553 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:58.754505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cbaf2-9153-47a2-828f-00a931533553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:58.833908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-460cbaf2-9153-47a2-828f-00a931533553 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:34:58.835149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-460cbaf2-9153-47a2-828f-00a931533553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:58.946440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:34:59.099678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd299d6-5a0d-496a-aba6-2c8425606433 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:34:59.101257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bfd299d6-5a0d-496a-aba6-2c8425606433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:59.191364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:59.257704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd299d6-5a0d-496a-aba6-2c8425606433 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:34:59.387739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c42e11-b323-4119-b76d-cd6d1d88b299 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:34:59.388314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c42e11-b323-4119-b76d-cd6d1d88b299 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:34:59.388604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c42e11-b323-4119-b76d-cd6d1d88b299 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:34:59.388875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c42e11-b323-4119-b76d-cd6d1d88b299 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:34:59.390507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-69c42e11-b323-4119-b76d-cd6d1d88b299 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:34:59.438787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:34:59.485435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c42e11-b323-4119-b76d-cd6d1d88b299 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:34:59.487043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c42e11-b323-4119-b76d-cd6d1d88b299 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:34:59.505165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 719/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:00.315885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129b6f8-1bb6-4054-9d7e-6a6a4c48238c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:00.316435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129b6f8-1bb6-4054-9d7e-6a6a4c48238c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:00.316435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129b6f8-1bb6-4054-9d7e-6a6a4c48238c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:00.317754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:00.324127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb805c6-0f4f-49d4-9df0-0db3edd35caa None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:00.324882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb805c6-0f4f-49d4-9df0-0db3edd35caa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:00.325029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb805c6-0f4f-49d4-9df0-0db3edd35caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:00.405960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1bb805c6-0f4f-49d4-9df0-0db3edd35caa None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:00.407184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1bb805c6-0f4f-49d4-9df0-0db3edd35caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:00.506767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:00.662287 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129b6f8-1bb6-4054-9d7e-6a6a4c48238c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:00.664686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6129b6f8-1bb6-4054-9d7e-6a6a4c48238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:00.725016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:00.830109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129b6f8-1bb6-4054-9d7e-6a6a4c48238c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:00.952211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2452fa-b475-4a4f-b8ba-b8c5efed9262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:00.952776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2452fa-b475-4a4f-b8ba-b8c5efed9262 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:00.953056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2452fa-b475-4a4f-b8ba-b8c5efed9262 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:00.953336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2452fa-b475-4a4f-b8ba-b8c5efed9262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:00.955071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dc2452fa-b475-4a4f-b8ba-b8c5efed9262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:00.963596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:01.065712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2452fa-b475-4a4f-b8ba-b8c5efed9262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:01.066748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2452fa-b475-4a4f-b8ba-b8c5efed9262 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:01.085827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 711/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:01.874608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17adbf-c20b-4c6a-b832-ae981ec29a69 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:01.875206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17adbf-c20b-4c6a-b832-ae981ec29a69 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:01.875206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17adbf-c20b-4c6a-b832-ae981ec29a69 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:01.876623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:01.882599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338c704-06b9-4cc4-8435-3ce7a9782ba5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:01.883305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338c704-06b9-4cc4-8435-3ce7a9782ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:01.883476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338c704-06b9-4cc4-8435-3ce7a9782ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:01.962429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6338c704-06b9-4cc4-8435-3ce7a9782ba5 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:01.963711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6338c704-06b9-4cc4-8435-3ce7a9782ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:02.037216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:02.069873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:02.221316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17adbf-c20b-4c6a-b832-ae981ec29a69 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:02.223400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9d17adbf-c20b-4c6a-b832-ae981ec29a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:02.250585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:02.385750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17adbf-c20b-4c6a-b832-ae981ec29a69 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:02.478935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:02.505209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c26960-a287-43a6-8a7e-eefd10b4e2f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:02.505861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c26960-a287-43a6-8a7e-eefd10b4e2f7 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:02.506140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c26960-a287-43a6-8a7e-eefd10b4e2f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:02.506416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c26960-a287-43a6-8a7e-eefd10b4e2f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:02.508026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-03c26960-a287-43a6-8a7e-eefd10b4e2f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:02.603105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c26960-a287-43a6-8a7e-eefd10b4e2f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:02.604774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c26960-a287-43a6-8a7e-eefd10b4e2f7 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:02.623421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 722/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:03.416734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58202-7b7c-4552-9405-7a850d5c7ab2 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:03.417541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58202-7b7c-4552-9405-7a850d5c7ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:03.417541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58202-7b7c-4552-9405-7a850d5c7ab2 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:03.419841 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:35:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:03.426089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c50c1f-214f-47bd-af7c-319c061ccb06 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:03.426561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c50c1f-214f-47bd-af7c-319c061ccb06 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:03.426670 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c50c1f-214f-47bd-af7c-319c061ccb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:03.510180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-26c50c1f-214f-47bd-af7c-319c061ccb06 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:03.511411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-26c50c1f-214f-47bd-af7c-319c061ccb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:03.553999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:03.625213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:35:03.779715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58202-7b7c-4552-9405-7a850d5c7ab2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:03.782045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-12a58202-7b7c-4552-9405-7a850d5c7ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:03.966116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58202-7b7c-4552-9405-7a850d5c7ab2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:04.029931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:04.093694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf9aff-3a57-4216-b1c5-08eb1e46bbcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:04.094430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf9aff-3a57-4216-b1c5-08eb1e46bbcf service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:04.094715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf9aff-3a57-4216-b1c5-08eb1e46bbcf service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:04.094990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf9aff-3a57-4216-b1c5-08eb1e46bbcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:04.096687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b1cf9aff-3a57-4216-b1c5-08eb1e46bbcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:04.191090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf9aff-3a57-4216-b1c5-08eb1e46bbcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:04.192763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf9aff-3a57-4216-b1c5-08eb1e46bbcf service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:04.211821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 714/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:05.045809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c896f5-0503-4ce9-947d-c5f5ca93cc79 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:05.046348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c896f5-0503-4ce9-947d-c5f5ca93cc79 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:05.046348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c896f5-0503-4ce9-947d-c5f5ca93cc79 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:05.047911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:05.056041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41a38-9b08-4621-a829-fbcfa8a780a6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:05.056512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41a38-9b08-4621-a829-fbcfa8a780a6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:05.056655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41a38-9b08-4621-a829-fbcfa8a780a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:05.140183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1ba41a38-9b08-4621-a829-fbcfa8a780a6 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:05.141427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1ba41a38-9b08-4621-a829-fbcfa8a780a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:05.143520 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:05.244896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:05.398691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c896f5-0503-4ce9-947d-c5f5ca93cc79 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:05.400980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-02c896f5-0503-4ce9-947d-c5f5ca93cc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:05.553430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c896f5-0503-4ce9-947d-c5f5ca93cc79 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:05.582122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:05.673148 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b159120-9736-4a15-8d20-2c7a74b2e8e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:05.673711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b159120-9736-4a15-8d20-2c7a74b2e8e0 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:05.673992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b159120-9736-4a15-8d20-2c7a74b2e8e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:05.674264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b159120-9736-4a15-8d20-2c7a74b2e8e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:05.675945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3b159120-9736-4a15-8d20-2c7a74b2e8e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:05.766224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b159120-9736-4a15-8d20-2c7a74b2e8e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:05.767847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b159120-9736-4a15-8d20-2c7a74b2e8e0 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:05.785884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 725/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:07.704806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639381d-7ff8-4463-b301-0a247a70899f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:07.705711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639381d-7ff8-4463-b301-0a247a70899f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:07.705711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639381d-7ff8-4463-b301-0a247a70899f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:07.708129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:07.714509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d303e5f-5ae9-4961-8bf4-d136d9228169 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:07.715184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d303e5f-5ae9-4961-8bf4-d136d9228169 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:07.715320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d303e5f-5ae9-4961-8bf4-d136d9228169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:07.758952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:07.787757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7d303e5f-5ae9-4961-8bf4-d136d9228169 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:07.788967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7d303e5f-5ae9-4961-8bf4-d136d9228169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:07.890337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:07.932010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639381d-7ff8-4463-b301-0a247a70899f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:07.934417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d639381d-7ff8-4463-b301-0a247a70899f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:08.048883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639381d-7ff8-4463-b301-0a247a70899f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:08.087925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:08.131677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddbc58-7c42-455a-bbbf-29c915c152ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:08.131868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddbc58-7c42-455a-bbbf-29c915c152ae service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:08.132154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddbc58-7c42-455a-bbbf-29c915c152ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:08.132431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddbc58-7c42-455a-bbbf-29c915c152ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:08.134107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-99ddbc58-7c42-455a-bbbf-29c915c152ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:08.253703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddbc58-7c42-455a-bbbf-29c915c152ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:08.255170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddbc58-7c42-455a-bbbf-29c915c152ae service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:08.274693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 717/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:10.503630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581488a-cd12-4a2b-a717-3a2cc403d60a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:10.504571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581488a-cd12-4a2b-a717-3a2cc403d60a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:10.504571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581488a-cd12-4a2b-a717-3a2cc403d60a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:10.506741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:10.516587 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6678d1-96f2-4816-a164-e37d2bf09a77 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:10.517323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6678d1-96f2-4816-a164-e37d2bf09a77 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:10.517483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6678d1-96f2-4816-a164-e37d2bf09a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:10.587680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8f6678d1-96f2-4816-a164-e37d2bf09a77 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:10.588896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8f6678d1-96f2-4816-a164-e37d2bf09a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:10.601844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:10.697445 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:10.736340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581488a-cd12-4a2b-a717-3a2cc403d60a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:10.738782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d581488a-cd12-4a2b-a717-3a2cc403d60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:10.869983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581488a-cd12-4a2b-a717-3a2cc403d60a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:10.882538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:10.951756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86f214-a474-480f-bc65-06d0dc96e5d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:10.952654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86f214-a474-480f-bc65-06d0dc96e5d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:10.953028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86f214-a474-480f-bc65-06d0dc96e5d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:10.953474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86f214-a474-480f-bc65-06d0dc96e5d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:10.955134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bd86f214-a474-480f-bc65-06d0dc96e5d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:11.075933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86f214-a474-480f-bc65-06d0dc96e5d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:11.078385 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86f214-a474-480f-bc65-06d0dc96e5d6 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:11.098442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 728/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:13.429046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314d6de-ed58-423a-8eb1-5d7ecb241f6d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:13.429600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314d6de-ed58-423a-8eb1-5d7ecb241f6d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:13.429600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314d6de-ed58-423a-8eb1-5d7ecb241f6d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:13.430927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:13.437555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9996c-e4d6-4a21-960e-60242004a328 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:13.438197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9996c-e4d6-4a21-960e-60242004a328 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:13.438372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9996c-e4d6-4a21-960e-60242004a328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:13.489190 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:13.508940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a1d9996c-e4d6-4a21-960e-60242004a328 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:13.510162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a1d9996c-e4d6-4a21-960e-60242004a328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:13.626638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:13.669730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314d6de-ed58-423a-8eb1-5d7ecb241f6d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:13.672181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4314d6de-ed58-423a-8eb1-5d7ecb241f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:13.782381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314d6de-ed58-423a-8eb1-5d7ecb241f6d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:13.825241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:13.859400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cff4ec-9bf0-4835-b9cf-3d4e35ea470f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:13.860125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cff4ec-9bf0-4835-b9cf-3d4e35ea470f service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:13.860426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cff4ec-9bf0-4835-b9cf-3d4e35ea470f service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:13.860696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cff4ec-9bf0-4835-b9cf-3d4e35ea470f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:13.862322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a9cff4ec-9bf0-4835-b9cf-3d4e35ea470f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:13.975570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cff4ec-9bf0-4835-b9cf-3d4e35ea470f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:13.977701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cff4ec-9bf0-4835-b9cf-3d4e35ea470f service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:14.120923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 720/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:16.259952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a802b7-52f7-4162-aec4-11d3709d301d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:16.260791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a802b7-52f7-4162-aec4-11d3709d301d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:16.260791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a802b7-52f7-4162-aec4-11d3709d301d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:16.263033 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:16.269734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661efc42-e5a9-4557-ae5e-4a2c721e776e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:16.270177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661efc42-e5a9-4557-ae5e-4a2c721e776e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:16.270278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661efc42-e5a9-4557-ae5e-4a2c721e776e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:16.341286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-661efc42-e5a9-4557-ae5e-4a2c721e776e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:16.342403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-661efc42-e5a9-4557-ae5e-4a2c721e776e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:16.364688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:16.450924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:16.496241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a802b7-52f7-4162-aec4-11d3709d301d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:16.497795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-93a802b7-52f7-4162-aec4-11d3709d301d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:16.622963 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a802b7-52f7-4162-aec4-11d3709d301d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:16.644336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:16.701221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94290cb8-7b01-45ab-9aa3-789a58c4823b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:16.701870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94290cb8-7b01-45ab-9aa3-789a58c4823b service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:16.702156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94290cb8-7b01-45ab-9aa3-789a58c4823b service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:16.702437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94290cb8-7b01-45ab-9aa3-789a58c4823b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:16.704127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-94290cb8-7b01-45ab-9aa3-789a58c4823b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:16.823266 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94290cb8-7b01-45ab-9aa3-789a58c4823b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:16.824840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94290cb8-7b01-45ab-9aa3-789a58c4823b service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:16.845273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 731/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:18.893980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6bfd1-57a0-4df6-aaae-5a27cf85bc14 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:18.894861 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6bfd1-57a0-4df6-aaae-5a27cf85bc14 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:18.894861 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6bfd1-57a0-4df6-aaae-5a27cf85bc14 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:18.897156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:18.902703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200eb95-191e-48b7-8a79-3d5bd60cc255 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:18.903121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200eb95-191e-48b7-8a79-3d5bd60cc255 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:18.903245 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200eb95-191e-48b7-8a79-3d5bd60cc255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:18.976222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:18.979632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a200eb95-191e-48b7-8a79-3d5bd60cc255 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:18.981745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a200eb95-191e-48b7-8a79-3d5bd60cc255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:19.083326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:19.130152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6bfd1-57a0-4df6-aaae-5a27cf85bc14 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:19.132510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-19c6bfd1-57a0-4df6-aaae-5a27cf85bc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:19.244314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6bfd1-57a0-4df6-aaae-5a27cf85bc14 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:19.301121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:19.328177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8cada-fc5e-46ee-92d0-791bbe9b289c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:19.328810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8cada-fc5e-46ee-92d0-791bbe9b289c service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:19.329102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8cada-fc5e-46ee-92d0-791bbe9b289c service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:19.329361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8cada-fc5e-46ee-92d0-791bbe9b289c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:19.331006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b7b8cada-fc5e-46ee-92d0-791bbe9b289c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:19.467189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8cada-fc5e-46ee-92d0-791bbe9b289c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:19.468686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8cada-fc5e-46ee-92d0-791bbe9b289c service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:19.487441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 723/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:21.567194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9adf9-4ac9-4ecf-bf5d-2e3edf5a01be None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:21.568135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9adf9-4ac9-4ecf-bf5d-2e3edf5a01be None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:21.568135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9adf9-4ac9-4ecf-bf5d-2e3edf5a01be None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:21.570679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:21.577022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790637e-e85e-4a9f-84e3-b22bfbf1b921 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:21.577489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790637e-e85e-4a9f-84e3-b22bfbf1b921 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:21.577622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790637e-e85e-4a9f-84e3-b22bfbf1b921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:21.643968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3790637e-e85e-4a9f-84e3-b22bfbf1b921 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:21.645195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3790637e-e85e-4a9f-84e3-b22bfbf1b921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:21.675890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:21.745236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:21.787852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9adf9-4ac9-4ecf-bf5d-2e3edf5a01be None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:21.790232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-88f9adf9-4ac9-4ecf-bf5d-2e3edf5a01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:21.919067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9adf9-4ac9-4ecf-bf5d-2e3edf5a01be None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:21.950713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:21.997430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411feace-97c3-4c85-b005-08432ad90db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:21.998080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411feace-97c3-4c85-b005-08432ad90db7 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:21.998368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411feace-97c3-4c85-b005-08432ad90db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:21.998648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411feace-97c3-4c85-b005-08432ad90db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:22.000456 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-411feace-97c3-4c85-b005-08432ad90db7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:22.119350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411feace-97c3-4c85-b005-08432ad90db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:22.120853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411feace-97c3-4c85-b005-08432ad90db7 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:22.140307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 734/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:24.192570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07de6-245c-4afb-9fea-0f9d952b9ba7 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:24.193451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07de6-245c-4afb-9fea-0f9d952b9ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:24.193451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07de6-245c-4afb-9fea-0f9d952b9ba7 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:24.196104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:24.204822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a470a-b0dd-4f7f-b63a-35daf75e7025 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:24.205737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a470a-b0dd-4f7f-b63a-35daf75e7025 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:24.205737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a470a-b0dd-4f7f-b63a-35daf75e7025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:24.280354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-449a470a-b0dd-4f7f-b63a-35daf75e7025 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:24.281647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-449a470a-b0dd-4f7f-b63a-35daf75e7025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:24.290348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:24.386571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:24.424460 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e07de6-245c-4afb-9fea-0f9d952b9ba7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:24.426822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-94e07de6-245c-4afb-9fea-0f9d952b9ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:24.543987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e07de6-245c-4afb-9fea-0f9d952b9ba7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:24.602617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:24.622381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16180163-94b4-4edc-af60-9b41770010de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:24.623040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16180163-94b4-4edc-af60-9b41770010de service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:24.623329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16180163-94b4-4edc-af60-9b41770010de service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:24.623634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16180163-94b4-4edc-af60-9b41770010de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:24.625297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-16180163-94b4-4edc-af60-9b41770010de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:24.736445 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16180163-94b4-4edc-af60-9b41770010de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:24.737896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16180163-94b4-4edc-af60-9b41770010de service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:24.756877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 726/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:26.808466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d328f0-4aaf-4bd6-a730-ebfdbc0af8e8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:26.809260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d328f0-4aaf-4bd6-a730-ebfdbc0af8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:26.809435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d328f0-4aaf-4bd6-a730-ebfdbc0af8e8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:26.811660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:26.818007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52930157-79bc-4eeb-ad8d-d97dc86df3d3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:26.818538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52930157-79bc-4eeb-ad8d-d97dc86df3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:26.818709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52930157-79bc-4eeb-ad8d-d97dc86df3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:26.888496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-52930157-79bc-4eeb-ad8d-d97dc86df3d3 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:26.889718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-52930157-79bc-4eeb-ad8d-d97dc86df3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:26.929354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:26.991740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:27.031568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d328f0-4aaf-4bd6-a730-ebfdbc0af8e8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:27.033105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-70d328f0-4aaf-4bd6-a730-ebfdbc0af8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:27.143884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d328f0-4aaf-4bd6-a730-ebfdbc0af8e8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:27.187548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:27.225827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea7921-39c5-4a40-9698-52b3d8c559c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:27.226480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea7921-39c5-4a40-9698-52b3d8c559c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:27.226764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea7921-39c5-4a40-9698-52b3d8c559c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:27.227041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea7921-39c5-4a40-9698-52b3d8c559c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:27.228714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-24ea7921-39c5-4a40-9698-52b3d8c559c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:27.345411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea7921-39c5-4a40-9698-52b3d8c559c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:27.346811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea7921-39c5-4a40-9698-52b3d8c559c7 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:27.365378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 737/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:29.392014 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b7c1a-ee19-45f7-b5c4-68040b333bb6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:29.392606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b7c1a-ee19-45f7-b5c4-68040b333bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:29.392606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b7c1a-ee19-45f7-b5c4-68040b333bb6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:29.394232 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:29.399746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117c6e-2adf-439b-92db-ede0776a82f5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:29.400206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117c6e-2adf-439b-92db-ede0776a82f5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:29.400300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117c6e-2adf-439b-92db-ede0776a82f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:29.472797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-62117c6e-2adf-439b-92db-ede0776a82f5 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:29.474014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-62117c6e-2adf-439b-92db-ede0776a82f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:29.492676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:29.576173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:29.613578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b7c1a-ee19-45f7-b5c4-68040b333bb6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:29.615906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-463b7c1a-ee19-45f7-b5c4-68040b333bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:29.733119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b7c1a-ee19-45f7-b5c4-68040b333bb6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:29.808185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:29.820166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d38aa-c925-429e-8232-ee80befa29d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:29.821099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d38aa-c925-429e-8232-ee80befa29d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:29.821497 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d38aa-c925-429e-8232-ee80befa29d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:29.821886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d38aa-c925-429e-8232-ee80befa29d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:29.824142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a49d38aa-c925-429e-8232-ee80befa29d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:29.939883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d38aa-c925-429e-8232-ee80befa29d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:29.941411 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d38aa-c925-429e-8232-ee80befa29d0 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:29.959830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 729/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:32.020090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab39e0a-947c-4103-9ff2-9e6333871d6a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:32.020691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab39e0a-947c-4103-9ff2-9e6333871d6a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:32.020691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab39e0a-947c-4103-9ff2-9e6333871d6a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:32.022040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:32.028584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8690a3-af24-4faa-a327-3764a9ef90d2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:32.029306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8690a3-af24-4faa-a327-3764a9ef90d2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:32.029468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8690a3-af24-4faa-a327-3764a9ef90d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:32.104444 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5a8690a3-af24-4faa-a327-3764a9ef90d2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:32.105661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5a8690a3-af24-4faa-a327-3764a9ef90d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:32.163055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:32.214378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:32.252872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab39e0a-947c-4103-9ff2-9e6333871d6a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:32.254426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-eab39e0a-947c-4103-9ff2-9e6333871d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:32.365552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab39e0a-947c-4103-9ff2-9e6333871d6a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:32.427168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:32.458482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a55b-b9a5-4707-a498-2fa3fa7ab7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:32.459159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a55b-b9a5-4707-a498-2fa3fa7ab7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:32.459421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a55b-b9a5-4707-a498-2fa3fa7ab7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:32.459731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a55b-b9a5-4707-a498-2fa3fa7ab7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:32.461404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4327a55b-b9a5-4707-a498-2fa3fa7ab7a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:32.575944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327a55b-b9a5-4707-a498-2fa3fa7ab7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:32.577381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327a55b-b9a5-4707-a498-2fa3fa7ab7a1 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:32.605005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 740/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:42.212415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=22 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:42.601084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=66 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:52.126109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266d854-4fba-49b5-8176-b76cfd97b789 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:52.126686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266d854-4fba-49b5-8176-b76cfd97b789 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:52.126686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266d854-4fba-49b5-8176-b76cfd97b789 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:52.128137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 731/1471] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:52.134412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f586eb7-af8a-438f-be00-693febdb276a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:52.134872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f586eb7-af8a-438f-be00-693febdb276a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:52.135036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f586eb7-af8a-438f-be00-693febdb276a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:52.203643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1f586eb7-af8a-438f-be00-693febdb276a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:52.204844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1f586eb7-af8a-438f-be00-693febdb276a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:52.303912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 741/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:52.402214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad82eb-c734-4935-b556-81ad68c1eb61 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:52.402506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad82eb-c734-4935-b556-81ad68c1eb61 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:52.402791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad82eb-c734-4935-b556-81ad68c1eb61 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:52.404223 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 732/1473] 10.222.0.47 () {54 vars in 962 bytes} [Wed Jul 13 20:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:52.410116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc2ecb-f721-4d89-9cde-e3c7c59e7194 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:52.410468 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc2ecb-f721-4d89-9cde-e3c7c59e7194 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:52.410829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc2ecb-f721-4d89-9cde-e3c7c59e7194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:52.495162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-82cc2ecb-f721-4d89-9cde-e3c7c59e7194 None None] MFA Rules not processed for user `cc21faf2237d405bba03e5309d93629f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:52.496422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-82cc2ecb-f721-4d89-9cde-e3c7c59e7194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:52.519419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:52.580677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 742/1474] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Jul 13 20:35:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:52.588423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad82eb-c734-4935-b556-81ad68c1eb61 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:52.589964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-46ad82eb-c734-4935-b556-81ad68c1eb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:52.697647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad82eb-c734-4935-b556-81ad68c1eb61 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:52.774917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a9d0-9052-4990-9ed1-572fe1070610 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc21faf2237d405bba03e5309d93629f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:52.775611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a9d0-9052-4990-9ed1-572fe1070610 service glance] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:52.776353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a9d0-9052-4990-9ed1-572fe1070610 service glance] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:52.776353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a9d0-9052-4990-9ed1-572fe1070610 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:52.777806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8cb6a9d0-9052-4990-9ed1-572fe1070610 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:52.833502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:52.897825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6a9d0-9052-4990-9ed1-572fe1070610 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:52.899305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6a9d0-9052-4990-9ed1-572fe1070610 service glance] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:52.918140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 733/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:35:54.999205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522aecd-3f04-48b3-990b-7faca220e9ca None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:55.000004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522aecd-3f04-48b3-990b-7faca220e9ca None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:55.000099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522aecd-3f04-48b3-990b-7faca220e9ca None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:55.001469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 743/1476] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:35:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:55.009866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa206f-1a42-4551-b240-6aa72818eac0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:55.010599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa206f-1a42-4551-b240-6aa72818eac0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:55.010743 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa206f-1a42-4551-b240-6aa72818eac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:55.090563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b5aa206f-1a42-4551-b240-6aa72818eac0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:55.091806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b5aa206f-1a42-4551-b240-6aa72818eac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:55.195763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 734/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:35:55.219568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522aecd-3f04-48b3-990b-7faca220e9ca None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:55.221106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2522aecd-3f04-48b3-990b-7faca220e9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:55.350945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522aecd-3f04-48b3-990b-7faca220e9ca None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:55.403650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:55.458393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca784c-b08a-448f-a218-a16d65699e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:55.459084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca784c-b08a-448f-a218-a16d65699e87 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:55.459368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca784c-b08a-448f-a218-a16d65699e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:55.459685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca784c-b08a-448f-a218-a16d65699e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:55.460440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca784c-b08a-448f-a218-a16d65699e87 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:55.462172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-c9ca784c-b08a-448f-a218-a16d65699e87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:35:55.462172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 13 20:35:55.462172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: Jul 13 20:35:55.463315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca784c-b08a-448f-a218-a16d65699e87 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:55.482622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 744/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 13 20:35:55 2022] GET /identity/v3/projects => generated 2553 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:35:57.502333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583194ad-da39-4e37-9e46-cc7b0b7fa253 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:57.503183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583194ad-da39-4e37-9e46-cc7b0b7fa253 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:57.503183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583194ad-da39-4e37-9e46-cc7b0b7fa253 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:57.505400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 735/1479] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:35:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:35:57.513582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43cf6-6541-4a7c-b8ce-598bb3a48343 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:57.514016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43cf6-6541-4a7c-b8ce-598bb3a48343 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:57.514126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43cf6-6541-4a7c-b8ce-598bb3a48343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:57.591723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-94f43cf6-6541-4a7c-b8ce-598bb3a48343 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:35:57.592955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-94f43cf6-6541-4a7c-b8ce-598bb3a48343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:57.703823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 745/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:35:57.747707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583194ad-da39-4e37-9e46-cc7b0b7fa253 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:35:57.749734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-583194ad-da39-4e37-9e46-cc7b0b7fa253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:57.759791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:57.899559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583194ad-da39-4e37-9e46-cc7b0b7fa253 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:35:57.983097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:35:58.028403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77041d4-c7c1-4181-895e-93be9662327b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:35:58.029080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77041d4-c7c1-4181-895e-93be9662327b service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:35:58.029373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77041d4-c7c1-4181-895e-93be9662327b service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:35:58.029646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77041d4-c7c1-4181-895e-93be9662327b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:35:58.031817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b77041d4-c7c1-4181-895e-93be9662327b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:35:58.125466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77041d4-c7c1-4181-895e-93be9662327b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:35:58.127088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77041d4-c7c1-4181-895e-93be9662327b service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:35:58.145293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 736/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:36:00.350430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa7bdd-d69c-4859-b377-26a92721bfe8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:00.350964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa7bdd-d69c-4859-b377-26a92721bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:00.350964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa7bdd-d69c-4859-b377-26a92721bfe8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:00.352309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 746/1482] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:36:00.358660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0897ab3-01c0-46fb-bcd3-8364d230252e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:00.359281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0897ab3-01c0-46fb-bcd3-8364d230252e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:00.359387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0897ab3-01c0-46fb-bcd3-8364d230252e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:00.433522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a0897ab3-01c0-46fb-bcd3-8364d230252e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:36:00.434730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a0897ab3-01c0-46fb-bcd3-8364d230252e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:00.485296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:36:00.537153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 737/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:36:00.573902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa7bdd-d69c-4859-b377-26a92721bfe8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:36:00.576311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fdaa7bdd-d69c-4859-b377-26a92721bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:00.601375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:36:00.681496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa7bdd-d69c-4859-b377-26a92721bfe8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:36:00.758663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c5c38-9b50-4978-91a9-772d75b020a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:36:00.759321 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c5c38-9b50-4978-91a9-772d75b020a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:00.759638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c5c38-9b50-4978-91a9-772d75b020a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:00.759908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c5c38-9b50-4978-91a9-772d75b020a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:00.761550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1e3c5c38-9b50-4978-91a9-772d75b020a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:00.838037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:36:00.878801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c5c38-9b50-4978-91a9-772d75b020a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:36:00.880177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c5c38-9b50-4978-91a9-772d75b020a7 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:36:00.909109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 747/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:36:02.965173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e8e3d-2912-4c43-98f0-5be0523bfe02 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:02.965919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e8e3d-2912-4c43-98f0-5be0523bfe02 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:02.966100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e8e3d-2912-4c43-98f0-5be0523bfe02 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:02.968605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 738/1485] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:36:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:36:02.974203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d89067-17a1-45b2-b772-6208956ad982 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:02.974629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d89067-17a1-45b2-b772-6208956ad982 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:02.974765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d89067-17a1-45b2-b772-6208956ad982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:03.042212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f2d89067-17a1-45b2-b772-6208956ad982 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:36:03.043353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f2d89067-17a1-45b2-b772-6208956ad982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:03.146574 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 748/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:36:03.186537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e8e3d-2912-4c43-98f0-5be0523bfe02 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:36:03.188208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f35e8e3d-2912-4c43-98f0-5be0523bfe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:03.302902 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e8e3d-2912-4c43-98f0-5be0523bfe02 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:36:03.382679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:36:03.384640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86e21a-1a2c-4cd8-89d8-4688554d6277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:36:03.385226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86e21a-1a2c-4cd8-89d8-4688554d6277 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:03.385681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86e21a-1a2c-4cd8-89d8-4688554d6277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:03.385964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86e21a-1a2c-4cd8-89d8-4688554d6277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:03.387920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-af86e21a-1a2c-4cd8-89d8-4688554d6277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:03.506799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86e21a-1a2c-4cd8-89d8-4688554d6277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:36:03.508281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86e21a-1a2c-4cd8-89d8-4688554d6277 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:36:03.527369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 739/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:36:05.656057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab710aac-f295-4228-9fa3-ab1106f19617 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:05.656640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab710aac-f295-4228-9fa3-ab1106f19617 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:05.656640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab710aac-f295-4228-9fa3-ab1106f19617 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:05.658006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 749/1488] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:36:05.664218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1a2b5-db66-4119-a883-c2f94a623e2a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:05.664938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1a2b5-db66-4119-a883-c2f94a623e2a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:05.665036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1a2b5-db66-4119-a883-c2f94a623e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:05.745995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e3a1a2b5-db66-4119-a883-c2f94a623e2a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:36:05.747526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e3a1a2b5-db66-4119-a883-c2f94a623e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:05.809479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:36:05.856725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 740/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:36:05.900871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab710aac-f295-4228-9fa3-ab1106f19617 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:36:05.903306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ab710aac-f295-4228-9fa3-ab1106f19617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:05.950727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:36:06.025878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab710aac-f295-4228-9fa3-ab1106f19617 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:36:06.103044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c01c7e-eff6-4358-aa56-1f9dbef5fcf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:36:06.103734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c01c7e-eff6-4358-aa56-1f9dbef5fcf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:06.104028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c01c7e-eff6-4358-aa56-1f9dbef5fcf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:06.104304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c01c7e-eff6-4358-aa56-1f9dbef5fcf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:06.105958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-88c01c7e-eff6-4358-aa56-1f9dbef5fcf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:06.189026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:36:06.220270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c01c7e-eff6-4358-aa56-1f9dbef5fcf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:36:06.221715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c01c7e-eff6-4358-aa56-1f9dbef5fcf9 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:36:06.240395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 750/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:36:08.469503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b687f1-3fef-4f77-82cd-200cf98aa31a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:08.470040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b687f1-3fef-4f77-82cd-200cf98aa31a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:08.470153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b687f1-3fef-4f77-82cd-200cf98aa31a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:08.472076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 741/1491] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 13 20:36:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:36:08.480302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc622-8b84-449a-8e08-843c84d29fe0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:08.480698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc622-8b84-449a-8e08-843c84d29fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:08.480939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc622-8b84-449a-8e08-843c84d29fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:08.564422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-94bcc622-8b84-449a-8e08-843c84d29fe0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:36:08.565840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-94bcc622-8b84-449a-8e08-843c84d29fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:08.675539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 751/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 13 20:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:36:08.714209 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b687f1-3fef-4f77-82cd-200cf98aa31a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:36:08.716130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e6b687f1-3fef-4f77-82cd-200cf98aa31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:08.827315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b687f1-3fef-4f77-82cd-200cf98aa31a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:36:08.900330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96434357-ba75-4404-a04c-1f29ded3fe9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:36:08.901239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96434357-ba75-4404-a04c-1f29ded3fe9c service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:36:08.901526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96434357-ba75-4404-a04c-1f29ded3fe9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:36:08.901794 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96434357-ba75-4404-a04c-1f29ded3fe9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:36:08.903409 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-96434357-ba75-4404-a04c-1f29ded3fe9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:36:08.912418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:36:09.024807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96434357-ba75-4404-a04c-1f29ded3fe9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:36:09.026285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96434357-ba75-4404-a04c-1f29ded3fe9c service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:36:09.045376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 742/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:36:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:36:18.673510 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=82 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:36:19.042773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=50 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:42.278073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de57281-43c7-4a2c-b660-0ca67e25d158 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:42.278703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de57281-43c7-4a2c-b660-0ca67e25d158 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:42.278703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de57281-43c7-4a2c-b660-0ca67e25d158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:42.356856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1de57281-43c7-4a2c-b660-0ca67e25d158 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:37:42.358039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1de57281-43c7-4a2c-b660-0ca67e25d158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:42.458423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 752/1494] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:37:42.466818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857e039-e819-4b01-b9f1-4a430a6191cb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:42.467349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857e039-e819-4b01-b9f1-4a430a6191cb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:42.467349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857e039-e819-4b01-b9f1-4a430a6191cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:42.546938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d857e039-e819-4b01-b9f1-4a430a6191cb None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:37:42.548183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d857e039-e819-4b01-b9f1-4a430a6191cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:42.670473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 743/1495] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:37:42.675370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de57281-43c7-4a2c-b660-0ca67e25d158 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:42.676653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1de57281-43c7-4a2c-b660-0ca67e25d158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:42.688358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de57281-43c7-4a2c-b660-0ca67e25d158 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:42.770360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:42.806702 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:42.807371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:42.807712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:42.807990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:42.808707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:37:42.809063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:42.810027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:37:42.810027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 13 20:37:42.810027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: Jul 13 20:37:42.811025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:42.823506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 753/1496] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:37:42.831776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857e039-e819-4b01-b9f1-4a430a6191cb None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:42.841709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857e039-e819-4b01-b9f1-4a430a6191cb None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:42.907282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:42.943698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:42.944294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:42.944575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:42.944845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:42.945891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:42.947734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:37:42.947734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 13 20:37:42.947734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: Jul 13 20:37:42.948922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:43.005304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 744/1497] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:37:42 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:37:43.009688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:43.011206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:43.047339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e07e30-9876-4080-8874-c7859704883d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:43.150937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:43.151634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:43.151912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:43.152183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:43.152868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:43.153885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:37:43.153885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 13 20:37:43.153885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: Jul 13 20:37:43.155106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:43.198613 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 754/1498] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:37:43 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:37:43.204323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:43.205954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:43.274012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe26d77-c317-4aa4-97b0-0e41a52466d7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:43.370871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:43.386205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:43.387146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:43.387515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:43.387897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:43.388870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:43.389895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:37:43.389895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 13 20:37:43.389895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: Jul 13 20:37:43.390785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:43.403389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 745/1499] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:37:43.412189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:43.413724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:43.425762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fa68a-666a-4e1a-98ec-f1ea803543f6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:43.453433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:43.526810 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7ed84-acb5-4bac-bd1c-856c45ce5b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:43.527514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7ed84-acb5-4bac-bd1c-856c45ce5b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:43.527807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7ed84-acb5-4bac-bd1c-856c45ce5b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:43.528085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7ed84-acb5-4bac-bd1c-856c45ce5b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:43.528814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7ed84-acb5-4bac-bd1c-856c45ce5b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:43.529910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-14b7ed84-acb5-4bac-bd1c-856c45ce5b5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:37:43.529910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 13 20:37:43.529910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: Jul 13 20:37:43.530802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7ed84-acb5-4bac-bd1c-856c45ce5b5e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:43.543446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 755/1500] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:37:43.547875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:43.549422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:43.583277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3eb1b-a85d-4e88-96f7-a8633803d458 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:43.688677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a14fb-0329-4ff3-923d-80a10496236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:43.689576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a14fb-0329-4ff3-923d-80a10496236d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:43.689862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a14fb-0329-4ff3-923d-80a10496236d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:43.690131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a14fb-0329-4ff3-923d-80a10496236d admin admin] PATH_INFO: `/v3/projects/b321f38f998d468e9e47e414993d72f9/users/f1ff326a4a274e0abf876324083cac1e/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:43.713135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a14fb-0329-4ff3-923d-80a10496236d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b321f38f998d468e9e47e414993d72f9, user_id=f1ff326a4a274e0abf876324083cac1e, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:43.714178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-d69a14fb-0329-4ff3-923d-80a10496236d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:37:43.714178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 13 20:37:43.714178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: Jul 13 20:37:43.716150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a14fb-0329-4ff3-923d-80a10496236d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:43.742372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 746/1501] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:37:43 2022] PUT /identity/v3/projects/b321f38f998d468e9e47e414993d72f9/users/f1ff326a4a274e0abf876324083cac1e/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:37:43.754831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33886030-bfd2-4e64-8615-ff77aaf77665 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:43.755108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33886030-bfd2-4e64-8615-ff77aaf77665 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:43.755394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33886030-bfd2-4e64-8615-ff77aaf77665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:43.816594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-33886030-bfd2-4e64-8615-ff77aaf77665 None None] MFA Rules not processed for user `f1ff326a4a274e0abf876324083cac1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:37:43.818429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-33886030-bfd2-4e64-8615-ff77aaf77665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:43.904930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 756/1502] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:37:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:37:43.918737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89f7e2-bf8f-4e55-b23e-364667a0a1d3 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:43.919039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89f7e2-bf8f-4e55-b23e-364667a0a1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:43.919339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89f7e2-bf8f-4e55-b23e-364667a0a1d3 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:43.920835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 747/1503] 10.222.0.47 () {54 vars in 963 bytes} [Wed Jul 13 20:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:37:43.925759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9bf9b-2a42-4644-a2d1-a5c2cb2e54fe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:43.926061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9bf9b-2a42-4644-a2d1-a5c2cb2e54fe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:43.926340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9bf9b-2a42-4644-a2d1-a5c2cb2e54fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:43.958537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:44.005910 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-88d9bf9b-2a42-4644-a2d1-a5c2cb2e54fe None None] MFA Rules not processed for user `9fc23c141eff42f98d31deb2d8aaaca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:37:44.007061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-88d9bf9b-2a42-4644-a2d1-a5c2cb2e54fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:44.088731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 757/1504] 10.222.0.47 () {58 vars in 1060 bytes} [Wed Jul 13 20:37:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:37:44.096609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89f7e2-bf8f-4e55-b23e-364667a0a1d3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:44.098128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-eb89f7e2-bf8f-4e55-b23e-364667a0a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:44.152919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:44.209721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89f7e2-bf8f-4e55-b23e-364667a0a1d3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:44.285584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:44.286170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:44.286454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:44.286729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:44.288412 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:44.397663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:44.399156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:44.402890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:44.418814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 748/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:37:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:37:44.466963 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b4c71-458a-436b-ae4f-84ae2216b6c5 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:44.467480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b4c71-458a-436b-ae4f-84ae2216b6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:44.467593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b4c71-458a-436b-ae4f-84ae2216b6c5 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:44.468856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 758/1506] 10.222.0.47 () {52 vars in 829 bytes} [Wed Jul 13 20:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:37:44.472567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:44.473801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:44.576166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d651ed2-9a4f-45e2-ad53-40b08f8516b2 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:44.680459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:44.681681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:44.682079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:44.682490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] PATH_INFO: `/v3/users/f1ff326a4a274e0abf876324083cac1e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:44.702465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ff326a4a274e0abf876324083cac1e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:44.703549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:37:44.703549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 13 20:37:44.703549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: Jul 13 20:37:44.704764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:44.715064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:44.769696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ff326a4a274e0abf876324083cac1e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:44.770131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ff326a4a274e0abf876324083cac1e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:44.788451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ff326a4a274e0abf876324083cac1e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:44.802360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ff326a4a274e0abf876324083cac1e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:44.813763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ff326a4a274e0abf876324083cac1e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:44.821692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 749/1507] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:37:44 2022] DELETE /identity/v3/users/f1ff326a4a274e0abf876324083cac1e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:37:44.826977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce6262-002b-4576-9881-3bbaab4b90e8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:44.827209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce6262-002b-4576-9881-3bbaab4b90e8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:44.827522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce6262-002b-4576-9881-3bbaab4b90e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:44.884940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-29ce6262-002b-4576-9881-3bbaab4b90e8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:37:44.886153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-29ce6262-002b-4576-9881-3bbaab4b90e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:44.964440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:44.979980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 759/1508] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:37:44.990548 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:44.991836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:45.093926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0e382-0c66-4b67-b8ac-28851c468e2a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:45.148813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:45.170286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15b975-c174-4070-9e5e-65a8109519d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:45.170955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15b975-c174-4070-9e5e-65a8109519d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:45.171244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15b975-c174-4070-9e5e-65a8109519d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:45.171566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15b975-c174-4070-9e5e-65a8109519d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:45.173248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1b15b975-c174-4070-9e5e-65a8109519d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:45.294483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15b975-c174-4070-9e5e-65a8109519d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:45.296026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15b975-c174-4070-9e5e-65a8109519d5 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:45.315271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 750/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:37:45.694264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce6262-002b-4576-9881-3bbaab4b90e8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:37:45.695948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-29ce6262-002b-4576-9881-3bbaab4b90e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:37:45.707425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce6262-002b-4576-9881-3bbaab4b90e8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:37:45.814194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:37:45.815068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:37:45.815356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:37:45.815673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] PATH_INFO: `/v3/projects/b321f38f998d468e9e47e414993d72f9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:37:45.826635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b321f38f998d468e9e47e414993d72f9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:37:45.827713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:37:45.827713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 13 20:37:45.827713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: Jul 13 20:37:45.828906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:37:45.890452 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b321f38f998d468e9e47e414993d72f9', 'request_id': 'req-1cc524df-3515-4133-8fec-17fd9774ad25'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:45.904570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b321f38f998d468e9e47e414993d72f9', 'request_id': 'req-1cc524df-3515-4133-8fec-17fd9774ad25'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:45.911858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b321f38f998d468e9e47e414993d72f9', 'request_id': 'req-1cc524df-3515-4133-8fec-17fd9774ad25'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:45.944338 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] The token cache is being invalidate because project b321f38f998d468e9e47e414993d72f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:37:45.944621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:45.945045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1cc524df-3515-4133-8fec-17fd9774ad25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:37:45.946874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 760/1510] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:37:45 2022] DELETE /identity/v3/projects/b321f38f998d468e9e47e414993d72f9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:37:55.312754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=62 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:37:55.942380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:39:23.928965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc830a-e6f3-4026-a639-efcdb47ea7cf None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:23.929934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc830a-e6f3-4026-a639-efcdb47ea7cf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:23.929934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc830a-e6f3-4026-a639-efcdb47ea7cf None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:23.932149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:39:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:39:23.940303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abf26c-7bc3-48e5-9f30-f7583190c7f5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:23.940735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abf26c-7bc3-48e5-9f30-f7583190c7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:23.940887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abf26c-7bc3-48e5-9f30-f7583190c7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:24.034952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-69abf26c-7bc3-48e5-9f30-f7583190c7f5 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:24.036252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-69abf26c-7bc3-48e5-9f30-f7583190c7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:24.149238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:24.302991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc830a-e6f3-4026-a639-efcdb47ea7cf None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:39:24.306676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cedc830a-e6f3-4026-a639-efcdb47ea7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:24.449730 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc830a-e6f3-4026-a639-efcdb47ea7cf None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:39:24.456090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e0f8-8102-40b7-8396-6dc79f4a8c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:39:24.456745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e0f8-8102-40b7-8396-6dc79f4a8c53 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:24.457035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e0f8-8102-40b7-8396-6dc79f4a8c53 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:24.457310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e0f8-8102-40b7-8396-6dc79f4a8c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:24.466395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5e0f8-8102-40b7-8396-6dc79f4a8c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:39:24.468072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5e0f8-8102-40b7-8396-6dc79f4a8c53 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:39:24.478391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 752/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:39:25.312641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888704a-93ce-49a4-8b25-abd15a601485 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:25.313363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888704a-93ce-49a4-8b25-abd15a601485 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:25.313363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888704a-93ce-49a4-8b25-abd15a601485 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:25.314746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:39:25.323401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4d7bc-e031-4f7b-8fda-ad1d5689b1eb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:25.324208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4d7bc-e031-4f7b-8fda-ad1d5689b1eb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:25.324367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4d7bc-e031-4f7b-8fda-ad1d5689b1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:25.345322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-84c4d7bc-e031-4f7b-8fda-ad1d5689b1eb None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:25.346542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-84c4d7bc-e031-4f7b-8fda-ad1d5689b1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:25.355879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:25.518617 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888704a-93ce-49a4-8b25-abd15a601485 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:39:25.543494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888704a-93ce-49a4-8b25-abd15a601485 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:39:25.551306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde1e46-8673-4ef5-a111-f9cd52801b49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:39:25.552034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde1e46-8673-4ef5-a111-f9cd52801b49 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:25.552323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde1e46-8673-4ef5-a111-f9cd52801b49 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:25.552607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde1e46-8673-4ef5-a111-f9cd52801b49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:25.561777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde1e46-8673-4ef5-a111-f9cd52801b49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:39:25.563490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde1e46-8673-4ef5-a111-f9cd52801b49 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:39:25.574386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 763/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:39:25.830875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8684701-fce7-4354-9653-9b6564262db4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:25.831117 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8684701-fce7-4354-9653-9b6564262db4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:25.831335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8684701-fce7-4354-9653-9b6564262db4 None None] PATH_INFO: `/v3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:25.832733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 754/1517] 10.222.0.40 () {54 vars in 925 bytes} [Wed Jul 13 20:39:25 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:39:25.848800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7159-936e-4745-bac4-592e78656a1d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:25.849093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7159-936e-4745-bac4-592e78656a1d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:25.849370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7159-936e-4745-bac4-592e78656a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:25.928183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a72b7159-936e-4745-bac4-592e78656a1d None None] MFA Rules not processed for user `cbafea9ed6494a3f9aeba4140a32de4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:25.929427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a72b7159-936e-4745-bac4-592e78656a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:25.947367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:39:26.005079 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 764/1518] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Jul 13 20:39:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:35.340465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=56 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:39:36.001912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=23 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:39:36.227392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4a0e9-9744-4f9a-a96f-6dce3dc3949e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:36.227634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4a0e9-9744-4f9a-a96f-6dce3dc3949e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:36.227899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4a0e9-9744-4f9a-a96f-6dce3dc3949e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:36.229337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 755/1519] 10.222.0.47 () {54 vars in 960 bytes} [Wed Jul 13 20:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:39:36.234657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0ba64-38d2-4f6d-a05a-d06f8483ac41 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:36.234938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0ba64-38d2-4f6d-a05a-d06f8483ac41 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:36.235223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0ba64-38d2-4f6d-a05a-d06f8483ac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:36.253556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1af0ba64-38d2-4f6d-a05a-d06f8483ac41 None None] MFA Rules not processed for user `cbafea9ed6494a3f9aeba4140a32de4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:36.254751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1af0ba64-38d2-4f6d-a05a-d06f8483ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:36.263543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 765/1520] 10.222.0.47 () {58 vars in 1057 bytes} [Wed Jul 13 20:39:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:36.270910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4a0e9-9744-4f9a-a96f-6dce3dc3949e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:39:36.284868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4a0e9-9744-4f9a-a96f-6dce3dc3949e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:39:36.290108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7940d7c-d3ee-42e5-8e03-101f69ce5b4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:39:36.290820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7940d7c-d3ee-42e5-8e03-101f69ce5b4f service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:36.291172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7940d7c-d3ee-42e5-8e03-101f69ce5b4f service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:36.291545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7940d7c-d3ee-42e5-8e03-101f69ce5b4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:36.299914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7940d7c-d3ee-42e5-8e03-101f69ce5b4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:39:36.301539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7940d7c-d3ee-42e5-8e03-101f69ce5b4f service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:39:36.311266 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 756/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:39:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:39:36.837992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1669cc-7e41-4e9f-8a43-91c00ed5f933 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:36.838413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1669cc-7e41-4e9f-8a43-91c00ed5f933 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:36.838850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1669cc-7e41-4e9f-8a43-91c00ed5f933 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:36.841263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:39:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:39:36.847054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03d5b4-9bf8-47a4-acc4-308e0e056c74 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:36.847577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03d5b4-9bf8-47a4-acc4-308e0e056c74 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:36.847855 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03d5b4-9bf8-47a4-acc4-308e0e056c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:36.872952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fc03d5b4-9bf8-47a4-acc4-308e0e056c74 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:36.874070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fc03d5b4-9bf8-47a4-acc4-308e0e056c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:36.882784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:37.042367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1669cc-7e41-4e9f-8a43-91c00ed5f933 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:39:37.059425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1669cc-7e41-4e9f-8a43-91c00ed5f933 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:39:37.066027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfdd23-9941-42e1-b90c-dfad72ef2c7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:39:37.066155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfdd23-9941-42e1-b90c-dfad72ef2c7f service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:37.066402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfdd23-9941-42e1-b90c-dfad72ef2c7f service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:37.066678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfdd23-9941-42e1-b90c-dfad72ef2c7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:37.075972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfdd23-9941-42e1-b90c-dfad72ef2c7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:39:37.077594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfdd23-9941-42e1-b90c-dfad72ef2c7f service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:39:37.087261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 767/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:39:37.912997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430aadc-fa72-4648-847d-32197af317f1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:37.913619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430aadc-fa72-4648-847d-32197af317f1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:37.913619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430aadc-fa72-4648-847d-32197af317f1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:37.914971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:39:37.920081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ca12e-1d69-4029-a168-5c0c3a3ae748 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:37.920375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ca12e-1d69-4029-a168-5c0c3a3ae748 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:37.920655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ca12e-1d69-4029-a168-5c0c3a3ae748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:37.940758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-732ca12e-1d69-4029-a168-5c0c3a3ae748 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:37.941951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-732ca12e-1d69-4029-a168-5c0c3a3ae748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:37.950774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:38.103996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430aadc-fa72-4648-847d-32197af317f1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:39:38.119979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430aadc-fa72-4648-847d-32197af317f1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:39:38.125827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9024f-e58d-46ec-843e-30da7c908a40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:39:38.126479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9024f-e58d-46ec-843e-30da7c908a40 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:38.126750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9024f-e58d-46ec-843e-30da7c908a40 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:38.127028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9024f-e58d-46ec-843e-30da7c908a40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:38.135970 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9024f-e58d-46ec-843e-30da7c908a40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:39:38.137656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9024f-e58d-46ec-843e-30da7c908a40 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:39:38.147499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 759/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:39:42.064887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129d0df-53f5-4347-bf13-cde1e8e5ed93 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:42.065475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129d0df-53f5-4347-bf13-cde1e8e5ed93 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:42.065475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129d0df-53f5-4347-bf13-cde1e8e5ed93 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:42.067117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:39:42.073431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe1284-7d10-4394-926c-901d2dec13e3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:42.073871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe1284-7d10-4394-926c-901d2dec13e3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:42.074008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe1284-7d10-4394-926c-901d2dec13e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:42.096629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fdfe1284-7d10-4394-926c-901d2dec13e3 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:42.098019 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fdfe1284-7d10-4394-926c-901d2dec13e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:42.108520 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:42.185543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129d0df-53f5-4347-bf13-cde1e8e5ed93 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:39:42.187620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7129d0df-53f5-4347-bf13-cde1e8e5ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:42.277621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129d0df-53f5-4347-bf13-cde1e8e5ed93 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:39:42.285727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1fd76-05e9-4be7-af8f-f336e4b6a48e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:39:42.286951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1fd76-05e9-4be7-af8f-f336e4b6a48e service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:42.287253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1fd76-05e9-4be7-af8f-f336e4b6a48e service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:42.287570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1fd76-05e9-4be7-af8f-f336e4b6a48e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:42.299324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1fd76-05e9-4be7-af8f-f336e4b6a48e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:39:42.300877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1fd76-05e9-4be7-af8f-f336e4b6a48e service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:39:42.310948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 770/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:39:44.329463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a63352-1d2b-4feb-b799-03e3ed3330f7 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:44.330309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a63352-1d2b-4feb-b799-03e3ed3330f7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:44.330309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a63352-1d2b-4feb-b799-03e3ed3330f7 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:44.332328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:39:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:39:44.339161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d5391-d3e3-4775-8974-c0fdf42a2859 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:44.339712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d5391-d3e3-4775-8974-c0fdf42a2859 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:44.340131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d5391-d3e3-4775-8974-c0fdf42a2859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:44.363237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-750d5391-d3e3-4775-8974-c0fdf42a2859 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:44.364486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-750d5391-d3e3-4775-8974-c0fdf42a2859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:44.374305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:44.412163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a63352-1d2b-4feb-b799-03e3ed3330f7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:39:44.426302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a63352-1d2b-4feb-b799-03e3ed3330f7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:39:44.431332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e154b-e7c3-459b-9ff7-c76809c2093c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:39:44.432013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e154b-e7c3-459b-9ff7-c76809c2093c service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:44.432306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e154b-e7c3-459b-9ff7-c76809c2093c service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:44.432593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e154b-e7c3-459b-9ff7-c76809c2093c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:44.442992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e154b-e7c3-459b-9ff7-c76809c2093c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:39:44.444848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e154b-e7c3-459b-9ff7-c76809c2093c service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:39:44.454922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 762/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:39:47.999865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a7eae-2f0b-4b11-a7b5-e49d5f1d988c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:48.000469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a7eae-2f0b-4b11-a7b5-e49d5f1d988c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:48.000469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a7eae-2f0b-4b11-a7b5-e49d5f1d988c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:48.001826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:39:48.008403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a577719-53d6-4a06-8aed-694824e172e9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:48.008847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a577719-53d6-4a06-8aed-694824e172e9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:48.008990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a577719-53d6-4a06-8aed-694824e172e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:48.032156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3a577719-53d6-4a06-8aed-694824e172e9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:48.033368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3a577719-53d6-4a06-8aed-694824e172e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:48.042619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:48.347984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a7eae-2f0b-4b11-a7b5-e49d5f1d988c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:39:48.349971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b24a7eae-2f0b-4b11-a7b5-e49d5f1d988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:48.439084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a7eae-2f0b-4b11-a7b5-e49d5f1d988c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:39:48.443377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e94382-c7ed-4c95-9be4-06a18f239987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:39:48.444064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94382-c7ed-4c95-9be4-06a18f239987 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:48.444710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94382-c7ed-4c95-9be4-06a18f239987 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:48.444976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94382-c7ed-4c95-9be4-06a18f239987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:48.453769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e94382-c7ed-4c95-9be4-06a18f239987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:39:48.455231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e94382-c7ed-4c95-9be4-06a18f239987 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:39:48.465174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 773/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:39:50.485717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0d3f9-e7f5-4322-bcb5-fa6ada7d02c3 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:50.486561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0d3f9-e7f5-4322-bcb5-fa6ada7d02c3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:50.486561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0d3f9-e7f5-4322-bcb5-fa6ada7d02c3 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:50.489033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:39:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:39:50.497972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95fd78-6977-40d3-9778-9a12b4b19be1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:50.498411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95fd78-6977-40d3-9778-9a12b4b19be1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:50.498603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95fd78-6977-40d3-9778-9a12b4b19be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:50.520733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bb95fd78-6977-40d3-9778-9a12b4b19be1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:39:50.521954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bb95fd78-6977-40d3-9778-9a12b4b19be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:50.531026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:39:50.831410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0d3f9-e7f5-4322-bcb5-fa6ada7d02c3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:39:50.834174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e3e0d3f9-e7f5-4322-bcb5-fa6ada7d02c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:39:50.848706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0d3f9-e7f5-4322-bcb5-fa6ada7d02c3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:39:50.852367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87930879-73bc-441c-8e2b-67d518e2b418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:39:50.853026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930879-73bc-441c-8e2b-67d518e2b418 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:39:50.853314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930879-73bc-441c-8e2b-67d518e2b418 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:39:50.853595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930879-73bc-441c-8e2b-67d518e2b418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:39:50.862789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87930879-73bc-441c-8e2b-67d518e2b418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:39:50.864334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87930879-73bc-441c-8e2b-67d518e2b418 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:39:50.874043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 765/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:39:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:40:00.515509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:40:00.869568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=33 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:19.515733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff58b91-5411-4300-bf98-f3a56d3378a9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:19.516448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff58b91-5411-4300-bf98-f3a56d3378a9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:19.516448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff58b91-5411-4300-bf98-f3a56d3378a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:19.539975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fff58b91-5411-4300-bf98-f3a56d3378a9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:19.541190 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fff58b91-5411-4300-bf98-f3a56d3378a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:19.552161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 775/1540] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:19.561217 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488959e2-44a6-4895-b002-f2860b4149b0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:19.562784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488959e2-44a6-4895-b002-f2860b4149b0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:19.562784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488959e2-44a6-4895-b002-f2860b4149b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:19.584038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-488959e2-44a6-4895-b002-f2860b4149b0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:19.585372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-488959e2-44a6-4895-b002-f2860b4149b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:19.599296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 766/1541] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:19.604053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff58b91-5411-4300-bf98-f3a56d3378a9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:19.617403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff58b91-5411-4300-bf98-f3a56d3378a9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:19.625908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba98977-9b0e-4e81-8a23-924ab7f0acbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:19.626616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba98977-9b0e-4e81-8a23-924ab7f0acbe admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:19.627163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba98977-9b0e-4e81-8a23-924ab7f0acbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:19.627450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba98977-9b0e-4e81-8a23-924ab7f0acbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:19.628284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba98977-9b0e-4e81-8a23-924ab7f0acbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:19.628596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba98977-9b0e-4e81-8a23-924ab7f0acbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:19.630341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba98977-9b0e-4e81-8a23-924ab7f0acbe admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:19.647798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 776/1542] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:19.656108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488959e2-44a6-4895-b002-f2860b4149b0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:19.666959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488959e2-44a6-4895-b002-f2860b4149b0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:19.678192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66add14-0762-4d73-a1aa-e5f03ebd32f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:19.679341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66add14-0762-4d73-a1aa-e5f03ebd32f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:19.679905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66add14-0762-4d73-a1aa-e5f03ebd32f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:19.680556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66add14-0762-4d73-a1aa-e5f03ebd32f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:19.681498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66add14-0762-4d73-a1aa-e5f03ebd32f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:19.683759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66add14-0762-4d73-a1aa-e5f03ebd32f7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:19.741621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 767/1543] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:19 2022] POST /identity/v3/projects => generated 377 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:19.745430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba98977-9b0e-4e81-8a23-924ab7f0acbe admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:19.761845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba98977-9b0e-4e81-8a23-924ab7f0acbe admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:19.856465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e6719-633f-43fe-8ba1-f96724104a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:19.857067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e6719-633f-43fe-8ba1-f96724104a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:19.857354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e6719-633f-43fe-8ba1-f96724104a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:19.857632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e6719-633f-43fe-8ba1-f96724104a77 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:19.858339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e6719-633f-43fe-8ba1-f96724104a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:19.860311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e6719-633f-43fe-8ba1-f96724104a77 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:19.892471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 777/1544] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:19 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:19.896803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66add14-0762-4d73-a1aa-e5f03ebd32f7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:19.910662 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66add14-0762-4d73-a1aa-e5f03ebd32f7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:19.917868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc6be3-81d6-4824-833f-2310314fc522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:19.918900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc6be3-81d6-4824-833f-2310314fc522 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:19.919401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc6be3-81d6-4824-833f-2310314fc522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:19.919938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc6be3-81d6-4824-833f-2310314fc522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:19.920878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc6be3-81d6-4824-833f-2310314fc522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:19.922695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc6be3-81d6-4824-833f-2310314fc522 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:19.937122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 768/1545] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:19.941738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e6719-633f-43fe-8ba1-f96724104a77 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:19.958646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e6719-633f-43fe-8ba1-f96724104a77 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:19.968057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38422afb-314b-4e67-9018-9ef0b6439f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:19.968776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38422afb-314b-4e67-9018-9ef0b6439f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:19.969364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38422afb-314b-4e67-9018-9ef0b6439f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:19.969655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38422afb-314b-4e67-9018-9ef0b6439f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:19.970440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38422afb-314b-4e67-9018-9ef0b6439f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:19.972721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38422afb-314b-4e67-9018-9ef0b6439f12 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:19.994553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 778/1546] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:20.000195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc6be3-81d6-4824-833f-2310314fc522 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.016818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc6be3-81d6-4824-833f-2310314fc522 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.023429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a81b8-08d8-4ac7-9a3f-875256a3dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:20.024853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a81b8-08d8-4ac7-9a3f-875256a3dcd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.025374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a81b8-08d8-4ac7-9a3f-875256a3dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.026277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a81b8-08d8-4ac7-9a3f-875256a3dcd6 admin admin] PATH_INFO: `/v3/projects/3bb5a0bd02c6452597888d17874f760c/users/bf4d42cf5bec422b96f421b98c84491d/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.054237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a81b8-08d8-4ac7-9a3f-875256a3dcd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb5a0bd02c6452597888d17874f760c, user_id=bf4d42cf5bec422b96f421b98c84491d, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:20.057926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a81b8-08d8-4ac7-9a3f-875256a3dcd6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:20.086115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e9a0b-0eb7-4416-8099-c1b5860ee2ce None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.086468 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e9a0b-0eb7-4416-8099-c1b5860ee2ce None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.086678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e9a0b-0eb7-4416-8099-c1b5860ee2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.087357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 769/1547] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:19 2022] PUT /identity/v3/projects/3bb5a0bd02c6452597888d17874f760c/users/bf4d42cf5bec422b96f421b98c84491d/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:20.094919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e41969-fc26-4770-b8c0-f000e7d521c8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.095681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e41969-fc26-4770-b8c0-f000e7d521c8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.096187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e41969-fc26-4770-b8c0-f000e7d521c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.124038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e2e41969-fc26-4770-b8c0-f000e7d521c8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.125784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2e41969-fc26-4770-b8c0-f000e7d521c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.136799 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-249e9a0b-0eb7-4416-8099-c1b5860ee2ce None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.138028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-249e9a0b-0eb7-4416-8099-c1b5860ee2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.215362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 770/1548] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:20.235209 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e41969-fc26-4770-b8c0-f000e7d521c8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.332855 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e41969-fc26-4770-b8c0-f000e7d521c8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.342420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352fbf8-9f14-438d-b878-118a42399713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:20.343410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352fbf8-9f14-438d-b878-118a42399713 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.344177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352fbf8-9f14-438d-b878-118a42399713 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.345013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352fbf8-9f14-438d-b878-118a42399713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.357881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 779/1549] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.359804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbae86-e10a-4c81-bcfc-4dcba1d98a9b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.360665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbae86-e10a-4c81-bcfc-4dcba1d98a9b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.360665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbae86-e10a-4c81-bcfc-4dcba1d98a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.367603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352fbf8-9f14-438d-b878-118a42399713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:20.369524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352fbf8-9f14-438d-b878-118a42399713 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:20.403478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 771/1550] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:20.405426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6151-85e8-492c-82fc-3486e5c92f26 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.405994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6151-85e8-492c-82fc-3486e5c92f26 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.406495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6151-85e8-492c-82fc-3486e5c92f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.415364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f0bbae86-e10a-4c81-bcfc-4dcba1d98a9b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.416607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f0bbae86-e10a-4c81-bcfc-4dcba1d98a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.428074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ca3a6151-85e8-492c-82fc-3486e5c92f26 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.429774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ca3a6151-85e8-492c-82fc-3486e5c92f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.444522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 780/1551] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.446250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece39d2-9bc1-41ca-9faf-54b2647da3b1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.446539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece39d2-9bc1-41ca-9faf-54b2647da3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.446817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece39d2-9bc1-41ca-9faf-54b2647da3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.447799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 772/1552] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.455904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c29ab4-afbd-40c2-955f-0506ca72acd0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.456203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c29ab4-afbd-40c2-955f-0506ca72acd0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.456490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c29ab4-afbd-40c2-955f-0506ca72acd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.490720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fece39d2-9bc1-41ca-9faf-54b2647da3b1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.493817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fece39d2-9bc1-41ca-9faf-54b2647da3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.496097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-82c29ab4-afbd-40c2-955f-0506ca72acd0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.499165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-82c29ab4-afbd-40c2-955f-0506ca72acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.512186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 781/1553] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.516834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2797acc-8621-4d80-b71a-524e8503dbeb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.518856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2797acc-8621-4d80-b71a-524e8503dbeb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.518856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2797acc-8621-4d80-b71a-524e8503dbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.520525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 773/1554] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.523635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdb189-895f-4313-96a8-c0103a326860 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.525053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdb189-895f-4313-96a8-c0103a326860 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.525347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdb189-895f-4313-96a8-c0103a326860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.548968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f2797acc-8621-4d80-b71a-524e8503dbeb None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.550356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f2797acc-8621-4d80-b71a-524e8503dbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.551047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-68bdb189-895f-4313-96a8-c0103a326860 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.552524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-68bdb189-895f-4313-96a8-c0103a326860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.562118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 774/1555] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.568913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d434a6d-6352-4de0-be51-dab877d3ca11 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.568913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d434a6d-6352-4de0-be51-dab877d3ca11 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.568913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d434a6d-6352-4de0-be51-dab877d3ca11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.569214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 782/1556] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.569214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2797acc-8621-4d80-b71a-524e8503dbeb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.581909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2797acc-8621-4d80-b71a-524e8503dbeb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.588545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a43865-f711-462a-8789-7322dae6fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:20.589452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3d434a6d-6352-4de0-be51-dab877d3ca11 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.589889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a43865-f711-462a-8789-7322dae6fb8f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.590189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a43865-f711-462a-8789-7322dae6fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.590477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a43865-f711-462a-8789-7322dae6fb8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.590661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3d434a6d-6352-4de0-be51-dab877d3ca11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.591709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a43865-f711-462a-8789-7322dae6fb8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:20.591709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a43865-f711-462a-8789-7322dae6fb8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:20.593186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a43865-f711-462a-8789-7322dae6fb8f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:20.602254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 775/1557] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.602254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91ae85-aa09-42a3-9185-1e4abb6caaf6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.602254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91ae85-aa09-42a3-9185-1e4abb6caaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.602901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91ae85-aa09-42a3-9185-1e4abb6caaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.606570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 783/1558] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:20.608104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680e7b-129d-454a-990b-cb7953944be2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.608997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680e7b-129d-454a-990b-cb7953944be2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.608997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680e7b-129d-454a-990b-cb7953944be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.612171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:20.627919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7c91ae85-aa09-42a3-9185-1e4abb6caaf6 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.629430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7c91ae85-aa09-42a3-9185-1e4abb6caaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.638184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d9680e7b-129d-454a-990b-cb7953944be2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.639273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 776/1559] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.640135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91ae85-aa09-42a3-9185-1e4abb6caaf6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.645866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d9680e7b-129d-454a-990b-cb7953944be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.653002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91ae85-aa09-42a3-9185-1e4abb6caaf6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.656410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238b37-11f5-4ede-b0a8-86099ca52960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:20.657089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238b37-11f5-4ede-b0a8-86099ca52960 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.657378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238b37-11f5-4ede-b0a8-86099ca52960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.657654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238b37-11f5-4ede-b0a8-86099ca52960 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.658453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238b37-11f5-4ede-b0a8-86099ca52960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:20.659022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238b37-11f5-4ede-b0a8-86099ca52960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:20.660137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-a5238b37-11f5-4ede-b0a8-86099ca52960 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:41:20.660137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 13 20:41:20.660137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: Jul 13 20:41:20.661028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238b37-11f5-4ede-b0a8-86099ca52960 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:20.670297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 784/1560] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.670297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eca3a7-b9ca-4277-8b9b-aaf521bbfc46 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.670297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eca3a7-b9ca-4277-8b9b-aaf521bbfc46 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.670297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eca3a7-b9ca-4277-8b9b-aaf521bbfc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.678793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 777/1561] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:20.680460 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b67df-6d28-4a64-8f48-66a1c54c7d23 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.680758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b67df-6d28-4a64-8f48-66a1c54c7d23 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.681044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b67df-6d28-4a64-8f48-66a1c54c7d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.692888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-59eca3a7-b9ca-4277-8b9b-aaf521bbfc46 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.694796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-59eca3a7-b9ca-4277-8b9b-aaf521bbfc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.701636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c76b67df-6d28-4a64-8f48-66a1c54c7d23 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.702882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c76b67df-6d28-4a64-8f48-66a1c54c7d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.704187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 785/1562] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.708587 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eca3a7-b9ca-4277-8b9b-aaf521bbfc46 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.715565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 778/1563] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.715565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b67df-6d28-4a64-8f48-66a1c54c7d23 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.721828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eca3a7-b9ca-4277-8b9b-aaf521bbfc46 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.727204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b67df-6d28-4a64-8f48-66a1c54c7d23 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.727346 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ad313-634f-4162-bed2-1f21a268ab2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:20.728065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ad313-634f-4162-bed2-1f21a268ab2a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.728354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ad313-634f-4162-bed2-1f21a268ab2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.728639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ad313-634f-4162-bed2-1f21a268ab2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.729384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ad313-634f-4162-bed2-1f21a268ab2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:20.729694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ad313-634f-4162-bed2-1f21a268ab2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:20.732326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ad313-634f-4162-bed2-1f21a268ab2a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:20.735103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0068cf2-5a7f-47d5-8983-92e7847e5485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:20.735876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0068cf2-5a7f-47d5-8983-92e7847e5485 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.736221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0068cf2-5a7f-47d5-8983-92e7847e5485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.736561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0068cf2-5a7f-47d5-8983-92e7847e5485 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.737396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0068cf2-5a7f-47d5-8983-92e7847e5485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:20.739368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0068cf2-5a7f-47d5-8983-92e7847e5485 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:20.742518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:20.752116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 786/1564] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:20.752116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ad313-634f-4162-bed2-1f21a268ab2a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.759390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ad313-634f-4162-bed2-1f21a268ab2a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.766015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26ca30-ada2-4607-acec-bf60882598b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:20.766993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26ca30-ada2-4607-acec-bf60882598b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.767413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26ca30-ada2-4607-acec-bf60882598b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.767735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26ca30-ada2-4607-acec-bf60882598b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.768545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26ca30-ada2-4607-acec-bf60882598b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:20.768859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26ca30-ada2-4607-acec-bf60882598b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:20.770780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26ca30-ada2-4607-acec-bf60882598b7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:20.784321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 779/1565] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:20.784321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b7f39-2f02-41d7-8adb-3e8fd2075faa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.784321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b7f39-2f02-41d7-8adb-3e8fd2075faa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.784321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b7f39-2f02-41d7-8adb-3e8fd2075faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.784773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 787/1566] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:20.785728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26ca30-ada2-4607-acec-bf60882598b7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.795592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26ca30-ada2-4607-acec-bf60882598b7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.806657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b29b7f39-2f02-41d7-8adb-3e8fd2075faa None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.808218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b29b7f39-2f02-41d7-8adb-3e8fd2075faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:20.862941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73227059-2f4d-4232-b24e-f7221b368a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:20.864177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73227059-2f4d-4232-b24e-f7221b368a0b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.864643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73227059-2f4d-4232-b24e-f7221b368a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.865097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73227059-2f4d-4232-b24e-f7221b368a0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.865973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73227059-2f4d-4232-b24e-f7221b368a0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:20.867203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-73227059-2f4d-4232-b24e-f7221b368a0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:41:20.867203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 13 20:41:20.867203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: Jul 13 20:41:20.868956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73227059-2f4d-4232-b24e-f7221b368a0b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:20.885332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 780/1567] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:20.886215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b7f39-2f02-41d7-8adb-3e8fd2075faa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.897069 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b7f39-2f02-41d7-8adb-3e8fd2075faa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.903010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d7e7f-3024-41c9-abd8-3ef323f026c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:20.903701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 788/1568] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:20.903816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7e7f-3024-41c9-abd8-3ef323f026c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.904031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7e7f-3024-41c9-abd8-3ef323f026c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.904316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7e7f-3024-41c9-abd8-3ef323f026c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.905064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d7e7f-3024-41c9-abd8-3ef323f026c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:20.905381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d7e7f-3024-41c9-abd8-3ef323f026c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:20.906471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc290de7-f933-422b-adda-e12c1de84125 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:20.907809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d7e7f-3024-41c9-abd8-3ef323f026c0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:20.908141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc290de7-f933-422b-adda-e12c1de84125 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:20.908610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc290de7-f933-422b-adda-e12c1de84125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:20.923471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 781/1569] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:20.924783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d7e7f-3024-41c9-abd8-3ef323f026c0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:20.935954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bc290de7-f933-422b-adda-e12c1de84125 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:20.936668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d7e7f-3024-41c9-abd8-3ef323f026c0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:20.937872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bc290de7-f933-422b-adda-e12c1de84125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:21.006252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 789/1570] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:21.006611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1460c25-0191-4f19-ba4f-cdc03f117d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.007252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc290de7-f933-422b-adda-e12c1de84125 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.007667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1460c25-0191-4f19-ba4f-cdc03f117d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.008110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1460c25-0191-4f19-ba4f-cdc03f117d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.008520 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1460c25-0191-4f19-ba4f-cdc03f117d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.009362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1460c25-0191-4f19-ba4f-cdc03f117d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.011634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1460c25-0191-4f19-ba4f-cdc03f117d2e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.019233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc290de7-f933-422b-adda-e12c1de84125 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.024448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881498f5-a53e-43cd-8c88-308fe7ae7e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.024964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881498f5-a53e-43cd-8c88-308fe7ae7e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.025259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881498f5-a53e-43cd-8c88-308fe7ae7e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.025541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881498f5-a53e-43cd-8c88-308fe7ae7e3c admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.026249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881498f5-a53e-43cd-8c88-308fe7ae7e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.028284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881498f5-a53e-43cd-8c88-308fe7ae7e3c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.045310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 782/1571] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:20 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.047597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1460c25-0191-4f19-ba4f-cdc03f117d2e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.054193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 790/1572] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.054984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881498f5-a53e-43cd-8c88-308fe7ae7e3c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.066219 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881498f5-a53e-43cd-8c88-308fe7ae7e3c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.096026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:21.125006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:21.133474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477285a-f946-4365-97f0-f6f5e53a96c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.134133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477285a-f946-4365-97f0-f6f5e53a96c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.134423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477285a-f946-4365-97f0-f6f5e53a96c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.134744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477285a-f946-4365-97f0-f6f5e53a96c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.135898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477285a-f946-4365-97f0-f6f5e53a96c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:21.136302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477285a-f946-4365-97f0-f6f5e53a96c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.137228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1460c25-0191-4f19-ba4f-cdc03f117d2e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.138591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477285a-f946-4365-97f0-f6f5e53a96c2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.143536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7bf18-6cc9-47b5-92d8-f3b1881a8dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.144291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7bf18-6cc9-47b5-92d8-f3b1881a8dbd admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.144590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7bf18-6cc9-47b5-92d8-f3b1881a8dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.144873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7bf18-6cc9-47b5-92d8-f3b1881a8dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.145617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7bf18-6cc9-47b5-92d8-f3b1881a8dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.147740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7bf18-6cc9-47b5-92d8-f3b1881a8dbd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.155648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 791/1573] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:21.155648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477285a-f946-4365-97f0-f6f5e53a96c2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.167767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477285a-f946-4365-97f0-f6f5e53a96c2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.173524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf70d84-a01d-4583-8f7f-38cdaa907ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.174715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf70d84-a01d-4583-8f7f-38cdaa907ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.174840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf70d84-a01d-4583-8f7f-38cdaa907ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.175382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf70d84-a01d-4583-8f7f-38cdaa907ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.176037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf70d84-a01d-4583-8f7f-38cdaa907ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.178227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf70d84-a01d-4583-8f7f-38cdaa907ca7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.180750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 783/1574] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.181692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7bf18-6cc9-47b5-92d8-f3b1881a8dbd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.206754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 792/1575] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/users => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.208075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf70d84-a01d-4583-8f7f-38cdaa907ca7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.219662 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf70d84-a01d-4583-8f7f-38cdaa907ca7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.260721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7bf18-6cc9-47b5-92d8-f3b1881a8dbd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.267044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb368fe4-50ea-4c0f-9082-6da911eebe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.267749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb368fe4-50ea-4c0f-9082-6da911eebe83 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.268096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb368fe4-50ea-4c0f-9082-6da911eebe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.268515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb368fe4-50ea-4c0f-9082-6da911eebe83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.269214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb368fe4-50ea-4c0f-9082-6da911eebe83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.271153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb368fe4-50ea-4c0f-9082-6da911eebe83 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.292186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bdcc3-757a-4789-8a31-1b40c40279b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.293364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bdcc3-757a-4789-8a31-1b40c40279b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.293804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bdcc3-757a-4789-8a31-1b40c40279b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.294467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bdcc3-757a-4789-8a31-1b40c40279b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.295714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bdcc3-757a-4789-8a31-1b40c40279b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:21.296173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bdcc3-757a-4789-8a31-1b40c40279b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.298739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bdcc3-757a-4789-8a31-1b40c40279b2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.305560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 784/1576] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.305560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb368fe4-50ea-4c0f-9082-6da911eebe83 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.318109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 793/1577] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:21.319903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bdcc3-757a-4789-8a31-1b40c40279b2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.332450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bdcc3-757a-4789-8a31-1b40c40279b2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.390341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb368fe4-50ea-4c0f-9082-6da911eebe83 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.398449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde90b2-0e67-474d-ba61-c9aec9fb1b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.399537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde90b2-0e67-474d-ba61-c9aec9fb1b49 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.400135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde90b2-0e67-474d-ba61-c9aec9fb1b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.400571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde90b2-0e67-474d-ba61-c9aec9fb1b49 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.401799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde90b2-0e67-474d-ba61-c9aec9fb1b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.403062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-5fde90b2-0e67-474d-ba61-c9aec9fb1b49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:41:21.403062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 13 20:41:21.403062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: Jul 13 20:41:21.405025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde90b2-0e67-474d-ba61-c9aec9fb1b49 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.434466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 785/1578] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/users => generated 295 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.434466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde90b2-0e67-474d-ba61-c9aec9fb1b49 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.441042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca231b-007c-470d-96c2-e1d32aa512d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.441762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca231b-007c-470d-96c2-e1d32aa512d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.442236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca231b-007c-470d-96c2-e1d32aa512d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.442531 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca231b-007c-470d-96c2-e1d32aa512d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.443628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca231b-007c-470d-96c2-e1d32aa512d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.445899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca231b-007c-470d-96c2-e1d32aa512d1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.451381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde90b2-0e67-474d-ba61-c9aec9fb1b49 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.457886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c25015-b814-42c9-bc9d-9f972570e79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.458791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 794/1579] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:21.459037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c25015-b814-42c9-bc9d-9f972570e79c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.459420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c25015-b814-42c9-bc9d-9f972570e79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.459769 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c25015-b814-42c9-bc9d-9f972570e79c admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.460547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c25015-b814-42c9-bc9d-9f972570e79c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.462799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c25015-b814-42c9-bc9d-9f972570e79c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.463263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca231b-007c-470d-96c2-e1d32aa512d1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.475573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca231b-007c-470d-96c2-e1d32aa512d1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.482103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2081c-d152-4255-889b-1dbe4b4d7883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.482966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2081c-d152-4255-889b-1dbe4b4d7883 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.483310 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2081c-d152-4255-889b-1dbe4b4d7883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.483700 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2081c-d152-4255-889b-1dbe4b4d7883 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.484480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2081c-d152-4255-889b-1dbe4b4d7883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.486679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2081c-d152-4255-889b-1dbe4b4d7883 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.501080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 786/1580] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.502688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c25015-b814-42c9-bc9d-9f972570e79c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.512575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:21.517314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 795/1581] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/users => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.518325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2081c-d152-4255-889b-1dbe4b4d7883 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.531911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2081c-d152-4255-889b-1dbe4b4d7883 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.584034 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c25015-b814-42c9-bc9d-9f972570e79c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.589532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:21.590845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb859717-30de-424f-b3df-6543a82da24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.592099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb859717-30de-424f-b3df-6543a82da24a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.592577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb859717-30de-424f-b3df-6543a82da24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.593076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb859717-30de-424f-b3df-6543a82da24a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.594149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb859717-30de-424f-b3df-6543a82da24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.596647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb859717-30de-424f-b3df-6543a82da24a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.612025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 787/1582] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:21.613607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb859717-30de-424f-b3df-6543a82da24a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.617625 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212996e3-00a1-47e0-b6d1-a4b8d340c590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.618444 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212996e3-00a1-47e0-b6d1-a4b8d340c590 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.618947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212996e3-00a1-47e0-b6d1-a4b8d340c590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.619342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212996e3-00a1-47e0-b6d1-a4b8d340c590 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.620191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212996e3-00a1-47e0-b6d1-a4b8d340c590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.622494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212996e3-00a1-47e0-b6d1-a4b8d340c590 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.626370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb859717-30de-424f-b3df-6543a82da24a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.631840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251ab70-d511-4362-b63f-55e7e7fb16fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.632574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251ab70-d511-4362-b63f-55e7e7fb16fa admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.633008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251ab70-d511-4362-b63f-55e7e7fb16fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.633298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251ab70-d511-4362-b63f-55e7e7fb16fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.634219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251ab70-d511-4362-b63f-55e7e7fb16fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.637565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251ab70-d511-4362-b63f-55e7e7fb16fa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.645816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 796/1583] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.646845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212996e3-00a1-47e0-b6d1-a4b8d340c590 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.657720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212996e3-00a1-47e0-b6d1-a4b8d340c590 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.663419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d7aca-7bbe-466c-93b7-994879b0aeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.664250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d7aca-7bbe-466c-93b7-994879b0aeb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.664638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d7aca-7bbe-466c-93b7-994879b0aeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.665020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d7aca-7bbe-466c-93b7-994879b0aeb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.665924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d7aca-7bbe-466c-93b7-994879b0aeb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.667676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d7aca-7bbe-466c-93b7-994879b0aeb8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.676634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 788/1584] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.676634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251ab70-d511-4362-b63f-55e7e7fb16fa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.679731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 797/1585] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:21.680819 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d7aca-7bbe-466c-93b7-994879b0aeb8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.692751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d7aca-7bbe-466c-93b7-994879b0aeb8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.755805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb9f59-d75d-4e0d-9213-0a3760183f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.756610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9f59-d75d-4e0d-9213-0a3760183f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.757335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9f59-d75d-4e0d-9213-0a3760183f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.757733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9f59-d75d-4e0d-9213-0a3760183f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.758547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb9f59-d75d-4e0d-9213-0a3760183f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.758902 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251ab70-d511-4362-b63f-55e7e7fb16fa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.760749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb9f59-d75d-4e0d-9213-0a3760183f58 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.766748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b27e19-242a-420a-8786-b31a1e2b07f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.767651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b27e19-242a-420a-8786-b31a1e2b07f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.768078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b27e19-242a-420a-8786-b31a1e2b07f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.768465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b27e19-242a-420a-8786-b31a1e2b07f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.769441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b27e19-242a-420a-8786-b31a1e2b07f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.771498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b27e19-242a-420a-8786-b31a1e2b07f8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.790286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 789/1586] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:21.790286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b27e19-242a-420a-8786-b31a1e2b07f8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.793645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 798/1587] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.794682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb9f59-d75d-4e0d-9213-0a3760183f58 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.800808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b27e19-242a-420a-8786-b31a1e2b07f8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.806393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6532590-5029-429a-b3f8-df9e342b5d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.807029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6532590-5029-429a-b3f8-df9e342b5d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.807167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb9f59-d75d-4e0d-9213-0a3760183f58 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.807513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6532590-5029-429a-b3f8-df9e342b5d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.807891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6532590-5029-429a-b3f8-df9e342b5d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.808840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6532590-5029-429a-b3f8-df9e342b5d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.810596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6532590-5029-429a-b3f8-df9e342b5d40 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.812450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8341a89-d75f-4af6-8185-834e4883f32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.813653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8341a89-d75f-4af6-8185-834e4883f32a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.814151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8341a89-d75f-4af6-8185-834e4883f32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.814629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8341a89-d75f-4af6-8185-834e4883f32a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.815364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8341a89-d75f-4af6-8185-834e4883f32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.820999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8341a89-d75f-4af6-8185-834e4883f32a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.829466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 790/1588] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:21.831033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6532590-5029-429a-b3f8-df9e342b5d40 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.840314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 799/1589] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:21.840314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8341a89-d75f-4af6-8185-834e4883f32a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.852981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8341a89-d75f-4af6-8185-834e4883f32a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.853989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6532590-5029-429a-b3f8-df9e342b5d40 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.858395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d864b-27f8-46b5-a5a0-79561780bee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.859199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d864b-27f8-46b5-a5a0-79561780bee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.859673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d864b-27f8-46b5-a5a0-79561780bee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.860046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d864b-27f8-46b5-a5a0-79561780bee6 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.860560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c5a29-fae7-4ebd-99a5-51eab001b853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.860948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d864b-27f8-46b5-a5a0-79561780bee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.861263 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c5a29-fae7-4ebd-99a5-51eab001b853 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.863153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d864b-27f8-46b5-a5a0-79561780bee6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.863482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c5a29-fae7-4ebd-99a5-51eab001b853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.863960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c5a29-fae7-4ebd-99a5-51eab001b853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.865018 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c5a29-fae7-4ebd-99a5-51eab001b853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.867066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c5a29-fae7-4ebd-99a5-51eab001b853 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.882676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 791/1590] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:21.882676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c5a29-fae7-4ebd-99a5-51eab001b853 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.895162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 800/1591] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:21 2022] POST /identity/v3/users => generated 296 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:21.896587 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d864b-27f8-46b5-a5a0-79561780bee6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.900076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c5a29-fae7-4ebd-99a5-51eab001b853 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.907501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d526bc0-f06c-49bd-ad11-6520ee6883ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.908237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d526bc0-f06c-49bd-ad11-6520ee6883ef admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.908358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d864b-27f8-46b5-a5a0-79561780bee6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.908481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d526bc0-f06c-49bd-ad11-6520ee6883ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.908835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d526bc0-f06c-49bd-ad11-6520ee6883ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.909756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d526bc0-f06c-49bd-ad11-6520ee6883ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.911443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d526bc0-f06c-49bd-ad11-6520ee6883ef admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.913275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d017ad2-f034-49ed-84a1-8817a52f339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.914046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d017ad2-f034-49ed-84a1-8817a52f339f admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.914515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d017ad2-f034-49ed-84a1-8817a52f339f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.914839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d017ad2-f034-49ed-84a1-8817a52f339f admin admin] PATH_INFO: `/v3/projects/685e343103a7411f8c98e385fb866e2b/users/1807bba7ce074f8684ca48569832d8d1/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.927962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 792/1592] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:21.932558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d526bc0-f06c-49bd-ad11-6520ee6883ef admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.939096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d017ad2-f034-49ed-84a1-8817a52f339f admin admin] RBAC: Authorizing `identity:create_grant(project_id=685e343103a7411f8c98e385fb866e2b, user_id=1807bba7ce074f8684ca48569832d8d1, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.941787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-3d017ad2-f034-49ed-84a1-8817a52f339f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:41:21.941787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 13 20:41:21.941787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: Jul 13 20:41:21.945382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d017ad2-f034-49ed-84a1-8817a52f339f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.952837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d526bc0-f06c-49bd-ad11-6520ee6883ef admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:21.963484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 801/1593] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:21 2022] PUT /identity/v3/projects/685e343103a7411f8c98e385fb866e2b/users/1807bba7ce074f8684ca48569832d8d1/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:21.964350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d017ad2-f034-49ed-84a1-8817a52f339f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:21.966283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed27513-2a7d-4369-8ca9-a2b20a4649ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:21.966971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed27513-2a7d-4369-8ca9-a2b20a4649ed admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:21.967402 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed27513-2a7d-4369-8ca9-a2b20a4649ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:21.967705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed27513-2a7d-4369-8ca9-a2b20a4649ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:21.968583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed27513-2a7d-4369-8ca9-a2b20a4649ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:21.970217 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed27513-2a7d-4369-8ca9-a2b20a4649ed admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:21.993622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 793/1594] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:21.994540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed27513-2a7d-4369-8ca9-a2b20a4649ed admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.005145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed27513-2a7d-4369-8ca9-a2b20a4649ed admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.053878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:22.074233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d017ad2-f034-49ed-84a1-8817a52f339f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.081239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3459-55e9-4c8f-aa77-317bcfb8ee4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.082189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3459-55e9-4c8f-aa77-317bcfb8ee4a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.082791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3459-55e9-4c8f-aa77-317bcfb8ee4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.083240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3459-55e9-4c8f-aa77-317bcfb8ee4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.084280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c3459-55e9-4c8f-aa77-317bcfb8ee4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.087388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c3459-55e9-4c8f-aa77-317bcfb8ee4a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.090010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:22.097638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e478c-cdd2-4b14-b3c7-34d7c5e0f726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.100299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e478c-cdd2-4b14-b3c7-34d7c5e0f726 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.102064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e478c-cdd2-4b14-b3c7-34d7c5e0f726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.102462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e478c-cdd2-4b14-b3c7-34d7c5e0f726 admin admin] PATH_INFO: `/v3/projects/a81ec4c4754044b58723da1ce589b86c/users/3b4865ad7a6d43b1b13593a142c2dec1/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.104310 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 802/1595] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:22.105279 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3459-55e9-4c8f-aa77-317bcfb8ee4a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.119719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3459-55e9-4c8f-aa77-317bcfb8ee4a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.126511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9008ec-e93a-48ce-9dc3-c6918d820d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.127330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9008ec-e93a-48ce-9dc3-c6918d820d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.127776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9008ec-e93a-48ce-9dc3-c6918d820d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.128164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9008ec-e93a-48ce-9dc3-c6918d820d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.129027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9008ec-e93a-48ce-9dc3-c6918d820d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.131581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9008ec-e93a-48ce-9dc3-c6918d820d9b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.138983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e478c-cdd2-4b14-b3c7-34d7c5e0f726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a81ec4c4754044b58723da1ce589b86c, user_id=3b4865ad7a6d43b1b13593a142c2dec1, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.142121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e478c-cdd2-4b14-b3c7-34d7c5e0f726 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.149314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 803/1596] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:22.150432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9008ec-e93a-48ce-9dc3-c6918d820d9b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.159604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 794/1597] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:21 2022] PUT /identity/v3/projects/a81ec4c4754044b58723da1ce589b86c/users/3b4865ad7a6d43b1b13593a142c2dec1/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:22.160651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e478c-cdd2-4b14-b3c7-34d7c5e0f726 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.172047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9008ec-e93a-48ce-9dc3-c6918d820d9b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.245717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c0491-0416-4b8c-9c2b-4d78ccd9d063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.246423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c0491-0416-4b8c-9c2b-4d78ccd9d063 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.246704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c0491-0416-4b8c-9c2b-4d78ccd9d063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.247324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c0491-0416-4b8c-9c2b-4d78ccd9d063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.248273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c0491-0416-4b8c-9c2b-4d78ccd9d063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.250180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c0491-0416-4b8c-9c2b-4d78ccd9d063 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.259751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e478c-cdd2-4b14-b3c7-34d7c5e0f726 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.270217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 804/1598] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:22.270431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22116cde-e748-4717-a939-23c8be43ad7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.271164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22116cde-e748-4717-a939-23c8be43ad7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.271480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22116cde-e748-4717-a939-23c8be43ad7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.271659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d322c7-60bf-40a2-a9ab-f752877ae5b3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.271839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22116cde-e748-4717-a939-23c8be43ad7a admin admin] PATH_INFO: `/v3/projects/b29d4807663e4bfb934b2f7b3880f144/users/c54ec6918ad44a31ba7137abb56b89b8/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.271943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d322c7-60bf-40a2-a9ab-f752877ae5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.272214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d322c7-60bf-40a2-a9ab-f752877ae5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.291518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22116cde-e748-4717-a939-23c8be43ad7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29d4807663e4bfb934b2f7b3880f144, user_id=c54ec6918ad44a31ba7137abb56b89b8, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.294290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22116cde-e748-4717-a939-23c8be43ad7a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.305168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-81d322c7-60bf-40a2-a9ab-f752877ae5b3 None None] MFA Rules not processed for user `1807bba7ce074f8684ca48569832d8d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:22.306695 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-81d322c7-60bf-40a2-a9ab-f752877ae5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:22.316365 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 795/1599] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:22 2022] PUT /identity/v3/projects/b29d4807663e4bfb934b2f7b3880f144/users/c54ec6918ad44a31ba7137abb56b89b8/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:22.317342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22116cde-e748-4717-a939-23c8be43ad7a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.429543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 805/1600] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:22.429543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d322c7-60bf-40a2-a9ab-f752877ae5b3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.455040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22116cde-e748-4717-a939-23c8be43ad7a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.456195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d322c7-60bf-40a2-a9ab-f752877ae5b3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.456424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4eae54-55a2-4bf0-bfcf-c6925ca76a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.456424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4eae54-55a2-4bf0-bfcf-c6925ca76a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.456424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4eae54-55a2-4bf0-bfcf-c6925ca76a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.456424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4eae54-55a2-4bf0-bfcf-c6925ca76a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.456784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4eae54-55a2-4bf0-bfcf-c6925ca76a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.456784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4eae54-55a2-4bf0-bfcf-c6925ca76a1b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.473729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3835dd7-fc6d-481e-8ad2-56513b10af72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.474659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3835dd7-fc6d-481e-8ad2-56513b10af72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.474947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3835dd7-fc6d-481e-8ad2-56513b10af72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.475228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3835dd7-fc6d-481e-8ad2-56513b10af72 admin admin] PATH_INFO: `/v3/projects/94d7cd525d1c4af3a9dfc6d2fa7263ac/users/04df453b8eda42dd95e4a3dbc4b6e31f/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.481223 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 796/1601] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:22.481223 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4eae54-55a2-4bf0-bfcf-c6925ca76a1b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.507343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4eae54-55a2-4bf0-bfcf-c6925ca76a1b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.509640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3835dd7-fc6d-481e-8ad2-56513b10af72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94d7cd525d1c4af3a9dfc6d2fa7263ac, user_id=04df453b8eda42dd95e4a3dbc4b6e31f, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.513827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3835dd7-fc6d-481e-8ad2-56513b10af72 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.516269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1692d-b404-471f-8254-c418cef95d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.517199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1692d-b404-471f-8254-c418cef95d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.517489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1692d-b404-471f-8254-c418cef95d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.517776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1692d-b404-471f-8254-c418cef95d8e admin admin] PATH_INFO: `/v3/projects/84d603577ad942e2bfd7b14995dfc5ec/users/c501417991064faa85556c0f0651e99f/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.543520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 806/1602] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:22 2022] PUT /identity/v3/projects/94d7cd525d1c4af3a9dfc6d2fa7263ac/users/04df453b8eda42dd95e4a3dbc4b6e31f/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:22.543520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e5546-0294-4740-82c5-db82466d6bc6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.543520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e5546-0294-4740-82c5-db82466d6bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.544317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1692d-b404-471f-8254-c418cef95d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d603577ad942e2bfd7b14995dfc5ec, user_id=c501417991064faa85556c0f0651e99f, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.544424 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e5546-0294-4740-82c5-db82466d6bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.544509 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1692d-b404-471f-8254-c418cef95d8e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.559569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 797/1603] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:22 2022] PUT /identity/v3/projects/84d603577ad942e2bfd7b14995dfc5ec/users/c501417991064faa85556c0f0651e99f/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:22.559569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1692d-b404-471f-8254-c418cef95d8e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.559901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-444e5546-0294-4740-82c5-db82466d6bc6 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:22.559901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-444e5546-0294-4740-82c5-db82466d6bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:22.586830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:22.619170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:22.642408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 807/1604] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:22.644028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1740fb-6ab8-413d-b863-282382c3885d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.644379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1740fb-6ab8-413d-b863-282382c3885d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.644709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1740fb-6ab8-413d-b863-282382c3885d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.653023 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1692d-b404-471f-8254-c418cef95d8e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.658097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952a174-9376-48bf-af5d-098d5f5ce59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.659275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952a174-9376-48bf-af5d-098d5f5ce59c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.659275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952a174-9376-48bf-af5d-098d5f5ce59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.659711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952a174-9376-48bf-af5d-098d5f5ce59c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.661057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952a174-9376-48bf-af5d-098d5f5ce59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.662920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952a174-9376-48bf-af5d-098d5f5ce59c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.673160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 798/1605] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:22.673908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952a174-9376-48bf-af5d-098d5f5ce59c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.677320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4f1740fb-6ab8-413d-b863-282382c3885d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:22.678656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4f1740fb-6ab8-413d-b863-282382c3885d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:22.683942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952a174-9376-48bf-af5d-098d5f5ce59c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.687352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 808/1606] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:22.688868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cc01c-92f4-4e00-85f0-8c52195b55bc None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.689261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cc01c-92f4-4e00-85f0-8c52195b55bc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.689647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cc01c-92f4-4e00-85f0-8c52195b55bc None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.691886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd09117-b48a-4d75-8e3a-dfde78cd4be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.692993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd09117-b48a-4d75-8e3a-dfde78cd4be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.693541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 809/1607] 10.222.0.47 () {54 vars in 961 bytes} [Wed Jul 13 20:41:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:41:22.693973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd09117-b48a-4d75-8e3a-dfde78cd4be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.694412 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd09117-b48a-4d75-8e3a-dfde78cd4be9 admin admin] PATH_INFO: `/v3/projects/d006993d66cb4976b40858d4373f0e94/users/0b22e96941dd414da4fa22ed4bd36984/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.694887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b72568-27ed-4bfe-bec3-6326a5214e98 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.695172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b72568-27ed-4bfe-bec3-6326a5214e98 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.695452 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b72568-27ed-4bfe-bec3-6326a5214e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.712534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd09117-b48a-4d75-8e3a-dfde78cd4be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d006993d66cb4976b40858d4373f0e94, user_id=0b22e96941dd414da4fa22ed4bd36984, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.715239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd09117-b48a-4d75-8e3a-dfde78cd4be9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.724290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-80b72568-27ed-4bfe-bec3-6326a5214e98 None None] MFA Rules not processed for user `04df453b8eda42dd95e4a3dbc4b6e31f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:22.725506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-80b72568-27ed-4bfe-bec3-6326a5214e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:22.730644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 799/1608] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:22 2022] PUT /identity/v3/projects/d006993d66cb4976b40858d4373f0e94/users/0b22e96941dd414da4fa22ed4bd36984/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:22.732133 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d2f8-0c09-48a7-a2ee-e87e5ad1dfc6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.732426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d2f8-0c09-48a7-a2ee-e87e5ad1dfc6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.732839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d2f8-0c09-48a7-a2ee-e87e5ad1dfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.762456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3013d2f8-0c09-48a7-a2ee-e87e5ad1dfc6 None None] MFA Rules not processed for user `c501417991064faa85556c0f0651e99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:22.763756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3013d2f8-0c09-48a7-a2ee-e87e5ad1dfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:22.806389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 810/1609] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:22.807100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b72568-27ed-4bfe-bec3-6326a5214e98 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.858932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 800/1610] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:22.860288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d2f8-0c09-48a7-a2ee-e87e5ad1dfc6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:22.907636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b72568-27ed-4bfe-bec3-6326a5214e98 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.912552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e4bbc-8da4-4fcc-bd28-9fdb5a9ad79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.913370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4bbc-8da4-4fcc-bd28-9fdb5a9ad79a service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.914315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4bbc-8da4-4fcc-bd28-9fdb5a9ad79a service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.914540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4bbc-8da4-4fcc-bd28-9fdb5a9ad79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.935694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d2f8-0c09-48a7-a2ee-e87e5ad1dfc6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:22.940525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d90e6-7a4c-4090-9051-ba471f4fa659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:22.941431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d90e6-7a4c-4090-9051-ba471f4fa659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:22.941714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d90e6-7a4c-4090-9051-ba471f4fa659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:22.941991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d90e6-7a4c-4090-9051-ba471f4fa659 admin admin] PATH_INFO: `/v3/projects/670dd6acab474449a679024e1d357424/users/798de1406eac40b6bc5cf2f5bea4846f/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:22.961438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d90e6-7a4c-4090-9051-ba471f4fa659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670dd6acab474449a679024e1d357424, user_id=798de1406eac40b6bc5cf2f5bea4846f, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.988585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d90e6-7a4c-4090-9051-ba471f4fa659 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.989025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e4bbc-8da4-4fcc-bd28-9fdb5a9ad79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:22.990819 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e4bbc-8da4-4fcc-bd28-9fdb5a9ad79a service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:22.995045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:23.007585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 811/1611] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:23.010277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b4e04-5dae-4f30-893f-3f8d48b1e68f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.010753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b4e04-5dae-4f30-893f-3f8d48b1e68f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.011524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b4e04-5dae-4f30-893f-3f8d48b1e68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.023063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 801/1612] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:22 2022] PUT /identity/v3/projects/670dd6acab474449a679024e1d357424/users/798de1406eac40b6bc5cf2f5bea4846f/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:23.023994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d90e6-7a4c-4090-9051-ba471f4fa659 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:23.070373 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:23.076098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-331b4e04-5dae-4f30-893f-3f8d48b1e68f None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:23.078692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-331b4e04-5dae-4f30-893f-3f8d48b1e68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:23.164186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d90e6-7a4c-4090-9051-ba471f4fa659 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:23.175394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adfb6a-6c1e-4ec3-9dad-d9c6504173ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:23.176180 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adfb6a-6c1e-4ec3-9dad-d9c6504173ad service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.176462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adfb6a-6c1e-4ec3-9dad-d9c6504173ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.176738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adfb6a-6c1e-4ec3-9dad-d9c6504173ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.239516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 812/1613] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Jul 13 20:41:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:23.240817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86620dba-a70f-4906-9697-47798d755af7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.241105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86620dba-a70f-4906-9697-47798d755af7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.241387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86620dba-a70f-4906-9697-47798d755af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.296655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-86620dba-a70f-4906-9697-47798d755af7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:23.297897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-86620dba-a70f-4906-9697-47798d755af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:23.349833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adfb6a-6c1e-4ec3-9dad-d9c6504173ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:23.351362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adfb6a-6c1e-4ec3-9dad-d9c6504173ad service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:23.376601 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 802/1614] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:23.377614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adfb6a-6c1e-4ec3-9dad-d9c6504173ad service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:23.401427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adfb6a-6c1e-4ec3-9dad-d9c6504173ad service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:23.412630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fce6-b760-4d0c-8afa-3f24100c3c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:23.413316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fce6-b760-4d0c-8afa-3f24100c3c74 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.413602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fce6-b760-4d0c-8afa-3f24100c3c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.413880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fce6-b760-4d0c-8afa-3f24100c3c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.414637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16fce6-b760-4d0c-8afa-3f24100c3c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:23.419511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16fce6-b760-4d0c-8afa-3f24100c3c74 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:23.463756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 813/1615] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:23.464911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144e90-8587-4ce6-ac80-727806e8b76f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.465199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144e90-8587-4ce6-ac80-727806e8b76f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.465473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144e90-8587-4ce6-ac80-727806e8b76f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.466770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 814/1616] 10.222.0.47 () {54 vars in 961 bytes} [Wed Jul 13 20:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:41:23.481161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16041a6-4355-4d90-97d5-85a9c82bbcaa None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.481534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16041a6-4355-4d90-97d5-85a9c82bbcaa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.481802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16041a6-4355-4d90-97d5-85a9c82bbcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.491523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 803/1617] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:23 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:23.491962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fce6-b760-4d0c-8afa-3f24100c3c74 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:23.520709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f16041a6-4355-4d90-97d5-85a9c82bbcaa None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:23.521560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f16041a6-4355-4d90-97d5-85a9c82bbcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:23.619102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fce6-b760-4d0c-8afa-3f24100c3c74 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:23.634031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcd1ae-c976-4eda-8a8d-3ef5b199b36f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:23.634846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcd1ae-c976-4eda-8a8d-3ef5b199b36f service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.635872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcd1ae-c976-4eda-8a8d-3ef5b199b36f service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.636162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcd1ae-c976-4eda-8a8d-3ef5b199b36f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.668438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:23.680467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 815/1618] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:23.681909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1185d3-8311-4e9f-8838-be0ba5d15c59 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.682268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1185d3-8311-4e9f-8838-be0ba5d15c59 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.682612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1185d3-8311-4e9f-8838-be0ba5d15c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.718766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcd1ae-c976-4eda-8a8d-3ef5b199b36f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:23.720446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcd1ae-c976-4eda-8a8d-3ef5b199b36f service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:23.732402 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 804/1619] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:23.736591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7a1185d3-8311-4e9f-8838-be0ba5d15c59 None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:23.737942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7a1185d3-8311-4e9f-8838-be0ba5d15c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:23.745676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcd1ae-c976-4eda-8a8d-3ef5b199b36f service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:23.770373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcd1ae-c976-4eda-8a8d-3ef5b199b36f service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:23.782162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25e521-8026-4a5f-9fa1-ff7cc379e9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:23.783263 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25e521-8026-4a5f-9fa1-ff7cc379e9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.785076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25e521-8026-4a5f-9fa1-ff7cc379e9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.785469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25e521-8026-4a5f-9fa1-ff7cc379e9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.786744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 816/1620] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Jul 13 20:41:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:23.787108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25e521-8026-4a5f-9fa1-ff7cc379e9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:23.787710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1185d3-8311-4e9f-8838-be0ba5d15c59 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:23.792565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25e521-8026-4a5f-9fa1-ff7cc379e9c9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:23.843272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 805/1621] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:23 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:23.847387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25e521-8026-4a5f-9fa1-ff7cc379e9c9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:23.851338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab25e521-8026-4a5f-9fa1-ff7cc379e9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:23.865221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:23.907410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1185d3-8311-4e9f-8838-be0ba5d15c59 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:23.918559 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d52415-aea5-4155-81cf-ee17afb04fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:23.919243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d52415-aea5-4155-81cf-ee17afb04fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.919552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d52415-aea5-4155-81cf-ee17afb04fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:23.919834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d52415-aea5-4155-81cf-ee17afb04fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:23.944953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d52415-aea5-4155-81cf-ee17afb04fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:23.946415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d52415-aea5-4155-81cf-ee17afb04fd7 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:23.964499 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 817/1622] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:23.965744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d52415-aea5-4155-81cf-ee17afb04fd7 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:23.968960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25e521-8026-4a5f-9fa1-ff7cc379e9c9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:23.981950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3a01c-b1b0-47f4-bca3-2417ddb640a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:23.983300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3a01c-b1b0-47f4-bca3-2417ddb640a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:23.983946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3a01c-b1b0-47f4-bca3-2417ddb640a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.005201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3a01c-b1b0-47f4-bca3-2417ddb640a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.024972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d52415-aea5-4155-81cf-ee17afb04fd7 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:24.043276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3a01c-b1b0-47f4-bca3-2417ddb640a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:24.052802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3a01c-b1b0-47f4-bca3-2417ddb640a9 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:24.053007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55a65-7426-47f6-9462-812e5fffd989 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:24.053007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55a65-7426-47f6-9462-812e5fffd989 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.053223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55a65-7426-47f6-9462-812e5fffd989 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.056147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55a65-7426-47f6-9462-812e5fffd989 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.091036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 806/1623] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:24.092150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3a01c-b1b0-47f4-bca3-2417ddb640a9 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:24.102661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3a01c-b1b0-47f4-bca3-2417ddb640a9 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:24.111415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed315e2b-3318-4a8d-af2e-6f19b9fa44c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:24.112850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed315e2b-3318-4a8d-af2e-6f19b9fa44c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.113302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed315e2b-3318-4a8d-af2e-6f19b9fa44c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.113768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed315e2b-3318-4a8d-af2e-6f19b9fa44c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.114747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed315e2b-3318-4a8d-af2e-6f19b9fa44c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:24.117050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed315e2b-3318-4a8d-af2e-6f19b9fa44c5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:24.130077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 807/1624] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:24.131904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8f9eb-914e-42fe-b1db-3eefa0224b03 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.132348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8f9eb-914e-42fe-b1db-3eefa0224b03 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.132962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8f9eb-914e-42fe-b1db-3eefa0224b03 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.134927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 808/1625] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 13 20:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:41:24.135936 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8f9eb-914e-42fe-b1db-3eefa0224b03 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:24.151746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8f9eb-914e-42fe-b1db-3eefa0224b03 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:24.159181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cbff0-d5c8-45e0-97ac-7c0d4786b825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:24.160930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cbff0-d5c8-45e0-97ac-7c0d4786b825 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.163758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cbff0-d5c8-45e0-97ac-7c0d4786b825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.164293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cbff0-d5c8-45e0-97ac-7c0d4786b825 admin admin] PATH_INFO: `/v3/projects/edf525180db642c4b93b7a3e0df2de53/users/f037dbf4a6a74719989755cf73e0268f/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.202677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cbff0-d5c8-45e0-97ac-7c0d4786b825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf525180db642c4b93b7a3e0df2de53, user_id=f037dbf4a6a74719989755cf73e0268f, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:24.206888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cbff0-d5c8-45e0-97ac-7c0d4786b825 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:24.210916 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a55a65-7426-47f6-9462-812e5fffd989 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:24.213160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a55a65-7426-47f6-9462-812e5fffd989 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:24.237802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 809/1626] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:24 2022] PUT /identity/v3/projects/edf525180db642c4b93b7a3e0df2de53/users/f037dbf4a6a74719989755cf73e0268f/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:24.238876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 818/1627] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:24.242973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55a65-7426-47f6-9462-812e5fffd989 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:24.244234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3913017-505f-41de-b905-d82c97e46d17 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.245724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3913017-505f-41de-b905-d82c97e46d17 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.246256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3913017-505f-41de-b905-d82c97e46d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.264809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55a65-7426-47f6-9462-812e5fffd989 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:24.300638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e3913017-505f-41de-b905-d82c97e46d17 None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:24.308169 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e3913017-505f-41de-b905-d82c97e46d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:24.482806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d630f-1bee-40fc-b49a-6e0d5599e0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:24.485802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d630f-1bee-40fc-b49a-6e0d5599e0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.488756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d630f-1bee-40fc-b49a-6e0d5599e0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.489416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d630f-1bee-40fc-b49a-6e0d5599e0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.491354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d630f-1bee-40fc-b49a-6e0d5599e0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:24.495645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d630f-1bee-40fc-b49a-6e0d5599e0a1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:24.529275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 819/1628] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:24.530920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3579256-1d36-415e-a6eb-d6a906fe9cd0 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.532721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3579256-1d36-415e-a6eb-d6a906fe9cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.535727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3579256-1d36-415e-a6eb-d6a906fe9cd0 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.537781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 820/1629] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 13 20:41:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:41:24.538907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3579256-1d36-415e-a6eb-d6a906fe9cd0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:24.542508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 810/1630] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:41:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:24.546084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d87abe-30b1-49f3-84a6-96abca76bc27 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.546472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d87abe-30b1-49f3-84a6-96abca76bc27 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.546925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d87abe-30b1-49f3-84a6-96abca76bc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.561187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:24.572446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3579256-1d36-415e-a6eb-d6a906fe9cd0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:24.584265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196d79-23e7-491a-b3cf-43a96e35bcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:24.587875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196d79-23e7-491a-b3cf-43a96e35bcf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.589602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196d79-23e7-491a-b3cf-43a96e35bcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.589927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196d79-23e7-491a-b3cf-43a96e35bcf2 admin admin] PATH_INFO: `/v3/projects/edf525180db642c4b93b7a3e0df2de53/users/f037dbf4a6a74719989755cf73e0268f/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.594282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196d79-23e7-491a-b3cf-43a96e35bcf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf525180db642c4b93b7a3e0df2de53, user_id=f037dbf4a6a74719989755cf73e0268f, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:24.597282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f3d87abe-30b1-49f3-84a6-96abca76bc27 None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:24.600569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f3d87abe-30b1-49f3-84a6-96abca76bc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:24.601517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196d79-23e7-491a-b3cf-43a96e35bcf2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:24.621994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 811/1631] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:41:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:24.623969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d87abe-30b1-49f3-84a6-96abca76bc27 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:24.628568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 821/1632] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:24 2022] PUT /identity/v3/projects/edf525180db642c4b93b7a3e0df2de53/users/f037dbf4a6a74719989755cf73e0268f/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:24.639060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41534e58-6f27-4e72-a5fc-858e135d7f6d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.639060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41534e58-6f27-4e72-a5fc-858e135d7f6d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.639715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41534e58-6f27-4e72-a5fc-858e135d7f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.702672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-41534e58-6f27-4e72-a5fc-858e135d7f6d None None] MFA Rules not processed for user `f037dbf4a6a74719989755cf73e0268f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:24.705186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-41534e58-6f27-4e72-a5fc-858e135d7f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:24.750986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d87abe-30b1-49f3-84a6-96abca76bc27 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:24.789599 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:24.839843 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 822/1633] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:24 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:24.841809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41534e58-6f27-4e72-a5fc-858e135d7f6d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:24.867691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4239cf0-cfd2-4625-ba9b-8d9a0ecc8a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:24.868629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4239cf0-cfd2-4625-ba9b-8d9a0ecc8a5a service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:24.868982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4239cf0-cfd2-4625-ba9b-8d9a0ecc8a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:24.869290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4239cf0-cfd2-4625-ba9b-8d9a0ecc8a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:24.998247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41534e58-6f27-4e72-a5fc-858e135d7f6d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:25.006571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f771f-8878-4605-a62e-b6b13e926de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:25.007489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f771f-8878-4605-a62e-b6b13e926de9 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:25.010195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f771f-8878-4605-a62e-b6b13e926de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:25.010736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f771f-8878-4605-a62e-b6b13e926de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:25.012583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4239cf0-cfd2-4625-ba9b-8d9a0ecc8a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:25.014648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4239cf0-cfd2-4625-ba9b-8d9a0ecc8a5a service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:25.031229 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 812/1634] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:25.038482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f771f-8878-4605-a62e-b6b13e926de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:25.041783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f771f-8878-4605-a62e-b6b13e926de9 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:25.063517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 823/1635] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:25.957450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4239cf0-cfd2-4625-ba9b-8d9a0ecc8a5a service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:25.973745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4239cf0-cfd2-4625-ba9b-8d9a0ecc8a5a service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:25.982011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948824f8-a554-452e-9825-53c0cbc47e34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:25.984763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948824f8-a554-452e-9825-53c0cbc47e34 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:25.985127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948824f8-a554-452e-9825-53c0cbc47e34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:25.985901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948824f8-a554-452e-9825-53c0cbc47e34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:26.015276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948824f8-a554-452e-9825-53c0cbc47e34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:26.019397 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948824f8-a554-452e-9825-53c0cbc47e34 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:26.054673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 813/1636] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:25 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:26.988484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f771f-8878-4605-a62e-b6b13e926de9 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:27.099391 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f771f-8878-4605-a62e-b6b13e926de9 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:27.104925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:27.107218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:27.107669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:27.107975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] PATH_INFO: `/v3/users/04df453b8eda42dd95e4a3dbc4b6e31f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:27.149631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04df453b8eda42dd95e4a3dbc4b6e31f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:27.149631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:41:27.149631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 13 20:41:27.149631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: Jul 13 20:41:27.149631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:27.337663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04df453b8eda42dd95e4a3dbc4b6e31f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:27.354459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04df453b8eda42dd95e4a3dbc4b6e31f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:27.414222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04df453b8eda42dd95e4a3dbc4b6e31f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:27.437317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04df453b8eda42dd95e4a3dbc4b6e31f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:27.438415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e9c72290-0cd4-4e11-9a1c-7f0e02f97957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04df453b8eda42dd95e4a3dbc4b6e31f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:27.461936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 824/1637] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:26 2022] DELETE /identity/v3/users/04df453b8eda42dd95e4a3dbc4b6e31f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:27.466578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6d8f0-f05b-42bc-baec-c83b4497e3b0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:27.467778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6d8f0-f05b-42bc-baec-c83b4497e3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:27.470049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6d8f0-f05b-42bc-baec-c83b4497e3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:27.531089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-91c6d8f0-f05b-42bc-baec-c83b4497e3b0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:27.548782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-91c6d8f0-f05b-42bc-baec-c83b4497e3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:27.619874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77607f28-d628-42d5-a885-5f771def72a6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:27.620271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77607f28-d628-42d5-a885-5f771def72a6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:27.620569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77607f28-d628-42d5-a885-5f771def72a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:27.632827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:27.659440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:27.662745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-77607f28-d628-42d5-a885-5f771def72a6 None None] MFA Rules not processed for user `bf4d42cf5bec422b96f421b98c84491d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:27.664483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-77607f28-d628-42d5-a885-5f771def72a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:27.708073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 814/1638] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:27.715939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6d8f0-f05b-42bc-baec-c83b4497e3b0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:27.718519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-91c6d8f0-f05b-42bc-baec-c83b4497e3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:27.785127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 825/1639] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:27.787243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77607f28-d628-42d5-a885-5f771def72a6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:27.789432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-77607f28-d628-42d5-a885-5f771def72a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:27.819586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6d8f0-f05b-42bc-baec-c83b4497e3b0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:27.825442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c551d6e-b3df-4d51-b4c4-72e322932762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:27.826679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c551d6e-b3df-4d51-b4c4-72e322932762 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:27.827066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c551d6e-b3df-4d51-b4c4-72e322932762 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:27.827872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c551d6e-b3df-4d51-b4c4-72e322932762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:27.839399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c551d6e-b3df-4d51-b4c4-72e322932762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:27.842596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c551d6e-b3df-4d51-b4c4-72e322932762 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:27.857455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 815/1640] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:41:27.910160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77607f28-d628-42d5-a885-5f771def72a6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:27.922867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ddb3c-e4b5-4356-8717-8fe5be964129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:27.923593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ddb3c-e4b5-4356-8717-8fe5be964129 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:27.923887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ddb3c-e4b5-4356-8717-8fe5be964129 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:27.924294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ddb3c-e4b5-4356-8717-8fe5be964129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:27.935549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ddb3c-e4b5-4356-8717-8fe5be964129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:27.938160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ddb3c-e4b5-4356-8717-8fe5be964129 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:27.955531 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 826/1641] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:41:28.532088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c551d6e-b3df-4d51-b4c4-72e322932762 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:28.537179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1c551d6e-b3df-4d51-b4c4-72e322932762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:28.586963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c551d6e-b3df-4d51-b4c4-72e322932762 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:28.593886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:28.596825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:28.597143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:28.598076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] PATH_INFO: `/v3/projects/94d7cd525d1c4af3a9dfc6d2fa7263ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:28.618326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94d7cd525d1c4af3a9dfc6d2fa7263ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:28.625222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:41:28.625222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 13 20:41:28.625222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: Jul 13 20:41:28.636804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:28.779636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94d7cd525d1c4af3a9dfc6d2fa7263ac', 'request_id': 'req-d8846a05-1b3b-46be-b536-0f947636ee34'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:28.804727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94d7cd525d1c4af3a9dfc6d2fa7263ac', 'request_id': 'req-d8846a05-1b3b-46be-b536-0f947636ee34'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:28.822879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94d7cd525d1c4af3a9dfc6d2fa7263ac', 'request_id': 'req-d8846a05-1b3b-46be-b536-0f947636ee34'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:28.856069 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] The token cache is being invalidate because project 94d7cd525d1c4af3a9dfc6d2fa7263ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:28.856372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:28.857088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:28.868135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 816/1642] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:28 2022] DELETE /identity/v3/projects/94d7cd525d1c4af3a9dfc6d2fa7263ac => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:28.869233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ddb3c-e4b5-4356-8717-8fe5be964129 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:28.884655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ddb3c-e4b5-4356-8717-8fe5be964129 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:29.055551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:29.056995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:29.057673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:29.058097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] PATH_INFO: `/v3/users/f037dbf4a6a74719989755cf73e0268f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:29.059757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f037dbf4a6a74719989755cf73e0268f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:29.062007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:29.128101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:29.199288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f037dbf4a6a74719989755cf73e0268f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:29.200092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f037dbf4a6a74719989755cf73e0268f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:29.241397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f037dbf4a6a74719989755cf73e0268f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:29.269349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f037dbf4a6a74719989755cf73e0268f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:29.270121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-47a848a3-71b1-4d0a-a652-d4245c87ba35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f037dbf4a6a74719989755cf73e0268f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:29.326293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 827/1643] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:28 2022] DELETE /identity/v3/users/f037dbf4a6a74719989755cf73e0268f => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:30.392643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:30.394426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:30.419045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8846a05-1b3b-46be-b536-0f947636ee34 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:30.529103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:30.529894 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:30.530189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:30.530698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] PATH_INFO: `/v3/projects/edf525180db642c4b93b7a3e0df2de53` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:30.532433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf525180db642c4b93b7a3e0df2de53)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:30.534952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:30.545312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:30.610254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf525180db642c4b93b7a3e0df2de53', 'request_id': 'req-1542598f-9f60-4638-9564-869bbc1c1c59'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:30.624052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf525180db642c4b93b7a3e0df2de53', 'request_id': 'req-1542598f-9f60-4638-9564-869bbc1c1c59'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:30.633451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf525180db642c4b93b7a3e0df2de53', 'request_id': 'req-1542598f-9f60-4638-9564-869bbc1c1c59'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:30.648983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] The token cache is being invalidate because project edf525180db642c4b93b7a3e0df2de53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:30.649328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:30.650006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:30.657272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 817/1644] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:30 2022] DELETE /identity/v3/projects/edf525180db642c4b93b7a3e0df2de53 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:30.658291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e5c71-ed24-42af-b3cb-bd063df34553 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:30.658855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e5c71-ed24-42af-b3cb-bd063df34553 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:30.658953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e5c71-ed24-42af-b3cb-bd063df34553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:30.733831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-432e5c71-ed24-42af-b3cb-bd063df34553 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:30.739632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-432e5c71-ed24-42af-b3cb-bd063df34553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:30.754696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:30.756414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:30.863607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 828/1645] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:30.863896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542598f-9f60-4638-9564-869bbc1c1c59 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:30.872726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234d357-d381-43ca-963b-6bc3d97edbf6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:30.873014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234d357-d381-43ca-963b-6bc3d97edbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:30.873295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234d357-d381-43ca-963b-6bc3d97edbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:30.875244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:30.876191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:30.876485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:30.876765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] PATH_INFO: `/v3/users/1807bba7ce074f8684ca48569832d8d1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:30.878937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1807bba7ce074f8684ca48569832d8d1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:30.886007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:30.894608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e234d357-d381-43ca-963b-6bc3d97edbf6 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:30.895969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e234d357-d381-43ca-963b-6bc3d97edbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:30.907553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 829/1646] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:30.911087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234d357-d381-43ca-963b-6bc3d97edbf6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:30.929800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234d357-d381-43ca-963b-6bc3d97edbf6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:30.935940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba75cb7-53e4-4d5c-b4e8-cbdb76c52292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:30.936865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba75cb7-53e4-4d5c-b4e8-cbdb76c52292 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:30.937320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba75cb7-53e4-4d5c-b4e8-cbdb76c52292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:30.937755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba75cb7-53e4-4d5c-b4e8-cbdb76c52292 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:30.938667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba75cb7-53e4-4d5c-b4e8-cbdb76c52292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:30.939120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba75cb7-53e4-4d5c-b4e8-cbdb76c52292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:30.941591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba75cb7-53e4-4d5c-b4e8-cbdb76c52292 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:30.962214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 830/1647] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:30.963508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba75cb7-53e4-4d5c-b4e8-cbdb76c52292 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:30.967033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1807bba7ce074f8684ca48569832d8d1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:30.967858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1807bba7ce074f8684ca48569832d8d1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:30.980539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba75cb7-53e4-4d5c-b4e8-cbdb76c52292 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:30.986842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1807bba7ce074f8684ca48569832d8d1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:30.987679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3630d-2c72-4934-8bf8-b374f82f1087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:30.988367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3630d-2c72-4934-8bf8-b374f82f1087 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:30.988712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3630d-2c72-4934-8bf8-b374f82f1087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:30.988917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3630d-2c72-4934-8bf8-b374f82f1087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:30.989765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3630d-2c72-4934-8bf8-b374f82f1087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:30.992508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3630d-2c72-4934-8bf8-b374f82f1087 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:31.003838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1807bba7ce074f8684ca48569832d8d1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:31.017260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5bc8c5b-4f53-4e8d-a985-2392be3ee6d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1807bba7ce074f8684ca48569832d8d1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:31.034846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 818/1648] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:30 2022] DELETE /identity/v3/users/1807bba7ce074f8684ca48569832d8d1 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:31.036725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ada567-dbe0-4d07-b7a8-6c59482cbea2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:31.037128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ada567-dbe0-4d07-b7a8-6c59482cbea2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:31.037557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ada567-dbe0-4d07-b7a8-6c59482cbea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:31.039359 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 831/1649] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:30 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:31.046750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3630d-2c72-4934-8bf8-b374f82f1087 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:31.052849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-52a3630d-2c72-4934-8bf8-b374f82f1087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:31.095048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-56ada567-dbe0-4d07-b7a8-6c59482cbea2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:31.096879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-56ada567-dbe0-4d07-b7a8-6c59482cbea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:31.127495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:31.208211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:31.216826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3630d-2c72-4934-8bf8-b374f82f1087 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:31.234319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6759cd7-ac66-46d2-8d0b-34f4fb782561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:31.235254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6759cd7-ac66-46d2-8d0b-34f4fb782561 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:31.235970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6759cd7-ac66-46d2-8d0b-34f4fb782561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:31.236398 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6759cd7-ac66-46d2-8d0b-34f4fb782561 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:31.237372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6759cd7-ac66-46d2-8d0b-34f4fb782561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:31.239695 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6759cd7-ac66-46d2-8d0b-34f4fb782561 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:31.259696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 819/1650] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:31.303371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 832/1651] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:31 2022] POST /identity/v3/users => generated 283 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:31.306653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ada567-dbe0-4d07-b7a8-6c59482cbea2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:31.324079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ada567-dbe0-4d07-b7a8-6c59482cbea2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:31.328772 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4706e1-3731-4103-a529-4a6b751d6728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:31.329459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4706e1-3731-4103-a529-4a6b751d6728 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:31.329747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4706e1-3731-4103-a529-4a6b751d6728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:31.330028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4706e1-3731-4103-a529-4a6b751d6728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:31.330814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4706e1-3731-4103-a529-4a6b751d6728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:31.333302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4706e1-3731-4103-a529-4a6b751d6728 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:31.344267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 820/1652] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:31.348415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6759cd7-ac66-46d2-8d0b-34f4fb782561 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:31.350933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b6759cd7-ac66-46d2-8d0b-34f4fb782561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:31.358890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4706e1-3731-4103-a529-4a6b751d6728 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:31.381862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4706e1-3731-4103-a529-4a6b751d6728 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:31.394379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926feb4-3f7e-46ee-b2f9-3bfecd67e63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:31.395094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926feb4-3f7e-46ee-b2f9-3bfecd67e63d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:31.395094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926feb4-3f7e-46ee-b2f9-3bfecd67e63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:31.395288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926feb4-3f7e-46ee-b2f9-3bfecd67e63d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:31.396058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926feb4-3f7e-46ee-b2f9-3bfecd67e63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:31.398676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926feb4-3f7e-46ee-b2f9-3bfecd67e63d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:31.433964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 821/1653] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:31.440792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926feb4-3f7e-46ee-b2f9-3bfecd67e63d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:31.468502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926feb4-3f7e-46ee-b2f9-3bfecd67e63d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:31.483197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a731d0-c4da-401d-b5c0-b11b151c998c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:31.484477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a731d0-c4da-401d-b5c0-b11b151c998c admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:31.484769 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a731d0-c4da-401d-b5c0-b11b151c998c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:31.485051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a731d0-c4da-401d-b5c0-b11b151c998c admin admin] PATH_INFO: `/v3/projects/466c279cb08c4889a3fb42d913b46cfe/users/da7f349193ef497aa8944224103b75cd/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:31.494069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6759cd7-ac66-46d2-8d0b-34f4fb782561 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:31.502532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4868ff4-8a94-4e8a-873c-63bfc5deda11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:31.504968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4868ff4-8a94-4e8a-873c-63bfc5deda11 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:31.506454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4868ff4-8a94-4e8a-873c-63bfc5deda11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:31.507859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4868ff4-8a94-4e8a-873c-63bfc5deda11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:31.528626 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a731d0-c4da-401d-b5c0-b11b151c998c admin admin] RBAC: Authorizing `identity:create_grant(project_id=466c279cb08c4889a3fb42d913b46cfe, user_id=da7f349193ef497aa8944224103b75cd, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:31.532065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a731d0-c4da-401d-b5c0-b11b151c998c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:31.534039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4868ff4-8a94-4e8a-873c-63bfc5deda11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:31.542041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4868ff4-8a94-4e8a-873c-63bfc5deda11 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:31.573003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 822/1654] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:31 2022] PUT /identity/v3/projects/466c279cb08c4889a3fb42d913b46cfe/users/da7f349193ef497aa8944224103b75cd/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:31.574334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 833/1655] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:31.585900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1492a-a882-4678-9519-6d7f879bb7fb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:31.586172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1492a-a882-4678-9519-6d7f879bb7fb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:31.586461 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1492a-a882-4678-9519-6d7f879bb7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:31.622555 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0ee1492a-a882-4678-9519-6d7f879bb7fb None None] MFA Rules not processed for user `da7f349193ef497aa8944224103b75cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:31.624125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0ee1492a-a882-4678-9519-6d7f879bb7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:31.744374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 823/1656] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:31.746413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4868ff4-8a94-4e8a-873c-63bfc5deda11 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:31.759274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4868ff4-8a94-4e8a-873c-63bfc5deda11 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:31.832953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2fb68-37f5-4b10-84bb-7a6b8af2e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:31.834348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2fb68-37f5-4b10-84bb-7a6b8af2e00f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:31.834925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2fb68-37f5-4b10-84bb-7a6b8af2e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:31.835685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2fb68-37f5-4b10-84bb-7a6b8af2e00f admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:31.837305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2fb68-37f5-4b10-84bb-7a6b8af2e00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:31.839733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2fb68-37f5-4b10-84bb-7a6b8af2e00f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:31.880206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 834/1657] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:31 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:31.881192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1492a-a882-4678-9519-6d7f879bb7fb None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:31.898840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1492a-a882-4678-9519-6d7f879bb7fb None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:31.991538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078412c-156e-41ba-bf9b-c9ae12bc66c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:31.992249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078412c-156e-41ba-bf9b-c9ae12bc66c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:31.992542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078412c-156e-41ba-bf9b-c9ae12bc66c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:31.992827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078412c-156e-41ba-bf9b-c9ae12bc66c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:31.995893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078412c-156e-41ba-bf9b-c9ae12bc66c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:32.001679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078412c-156e-41ba-bf9b-c9ae12bc66c3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:32.020323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:32.046156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 824/1658] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:31 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:41:32.048831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2fb68-37f5-4b10-84bb-7a6b8af2e00f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:32.064667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2fb68-37f5-4b10-84bb-7a6b8af2e00f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:32.090368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991667e-cd97-4e79-8764-870afd521aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:32.091136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991667e-cd97-4e79-8764-870afd521aad admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:32.091794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991667e-cd97-4e79-8764-870afd521aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:32.092078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991667e-cd97-4e79-8764-870afd521aad admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:32.093906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991667e-cd97-4e79-8764-870afd521aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:32.101205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991667e-cd97-4e79-8764-870afd521aad admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:32.126443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078412c-156e-41ba-bf9b-c9ae12bc66c3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:32.127284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 835/1659] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:32.148299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078412c-156e-41ba-bf9b-c9ae12bc66c3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:32.159626 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40d0e8-ce82-4835-a852-9b53633f9bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:32.160763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40d0e8-ce82-4835-a852-9b53633f9bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:32.160869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40d0e8-ce82-4835-a852-9b53633f9bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:32.161102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40d0e8-ce82-4835-a852-9b53633f9bbc admin admin] PATH_INFO: `/v3/projects/854b3e18898941eb93d1bf64e9f41135/users/cec6cb434bb7493dbc2dbe368785fabe/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:32.186280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40d0e8-ce82-4835-a852-9b53633f9bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=854b3e18898941eb93d1bf64e9f41135, user_id=cec6cb434bb7493dbc2dbe368785fabe, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:32.189182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40d0e8-ce82-4835-a852-9b53633f9bbc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:32.211496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 825/1660] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:32 2022] PUT /identity/v3/projects/854b3e18898941eb93d1bf64e9f41135/users/cec6cb434bb7493dbc2dbe368785fabe/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:32.218186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991667e-cd97-4e79-8764-870afd521aad admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:32.230923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991667e-cd97-4e79-8764-870afd521aad admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:32.318335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f89c3-796d-44d2-8d1c-39e9b5a52620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:32.319478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f89c3-796d-44d2-8d1c-39e9b5a52620 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:32.320009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f89c3-796d-44d2-8d1c-39e9b5a52620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:32.320506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f89c3-796d-44d2-8d1c-39e9b5a52620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:32.321479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f89c3-796d-44d2-8d1c-39e9b5a52620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:32.323406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f89c3-796d-44d2-8d1c-39e9b5a52620 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:32.330296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:32.336709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 836/1661] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:32.340584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40d0e8-ce82-4835-a852-9b53633f9bbc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:32.362867 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40d0e8-ce82-4835-a852-9b53633f9bbc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:32.374089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdb9a0-3ac6-4663-b94d-e8ff5ff2a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:32.375911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdb9a0-3ac6-4663-b94d-e8ff5ff2a80e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:32.376477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdb9a0-3ac6-4663-b94d-e8ff5ff2a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:32.376979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdb9a0-3ac6-4663-b94d-e8ff5ff2a80e admin admin] PATH_INFO: `/v3/projects/854b3e18898941eb93d1bf64e9f41135/users/cec6cb434bb7493dbc2dbe368785fabe/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:32.379764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdb9a0-3ac6-4663-b94d-e8ff5ff2a80e admin admin] RBAC: Authorizing `identity:create_grant(project_id=854b3e18898941eb93d1bf64e9f41135, user_id=cec6cb434bb7493dbc2dbe368785fabe, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:32.383214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdb9a0-3ac6-4663-b94d-e8ff5ff2a80e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:32.404091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 826/1662] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:32 2022] PUT /identity/v3/projects/854b3e18898941eb93d1bf64e9f41135/users/cec6cb434bb7493dbc2dbe368785fabe/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:32.415014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d694e-6e93-45d0-a472-f6d6524933bb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:32.416101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d694e-6e93-45d0-a472-f6d6524933bb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:32.416642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d694e-6e93-45d0-a472-f6d6524933bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:32.462364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d01d694e-6e93-45d0-a472-f6d6524933bb None None] MFA Rules not processed for user `cec6cb434bb7493dbc2dbe368785fabe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:32.464569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d01d694e-6e93-45d0-a472-f6d6524933bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:32.575605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 837/1663] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:32 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:32.589414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdb9a0-3ac6-4663-b94d-e8ff5ff2a80e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:32.592549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-63fdb9a0-3ac6-4663-b94d-e8ff5ff2a80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:32.622154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d694e-6e93-45d0-a472-f6d6524933bb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:32.627768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d01d694e-6e93-45d0-a472-f6d6524933bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:32.815079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdb9a0-3ac6-4663-b94d-e8ff5ff2a80e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:32.835010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6de8b1-e295-4e43-b24f-1196b91bb93a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:32.837485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6de8b1-e295-4e43-b24f-1196b91bb93a service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:32.839389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6de8b1-e295-4e43-b24f-1196b91bb93a service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:32.841121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6de8b1-e295-4e43-b24f-1196b91bb93a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:32.881856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6de8b1-e295-4e43-b24f-1196b91bb93a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:32.883308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6de8b1-e295-4e43-b24f-1196b91bb93a service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:32.901617 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d694e-6e93-45d0-a472-f6d6524933bb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:32.915585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:32.916950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:32.917939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:32.918445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] PATH_INFO: `/v3/projects/685e343103a7411f8c98e385fb866e2b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:32.918908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 827/1664] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:32 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:32.920477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=685e343103a7411f8c98e385fb866e2b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:32.924387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:32.950691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38134678-f7d5-47ba-96b6-0db8bc7bb20d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:32.950971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38134678-f7d5-47ba-96b6-0db8bc7bb20d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:32.951733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38134678-f7d5-47ba-96b6-0db8bc7bb20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:32.977658 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-38134678-f7d5-47ba-96b6-0db8bc7bb20d None None] MFA Rules not processed for user `bf4d42cf5bec422b96f421b98c84491d`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:32.979318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-38134678-f7d5-47ba-96b6-0db8bc7bb20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:33.037125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '685e343103a7411f8c98e385fb866e2b', 'request_id': 'req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:33.076791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '685e343103a7411f8c98e385fb866e2b', 'request_id': 'req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:33.104585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '685e343103a7411f8c98e385fb866e2b', 'request_id': 'req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:33.148831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] The token cache is being invalidate because project 685e343103a7411f8c98e385fb866e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:33.150041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:33.151751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c79293c2-83ec-4f53-8e08-a4b45bcd7ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:33.156617 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 838/1665] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:32 2022] DELETE /identity/v3/projects/685e343103a7411f8c98e385fb866e2b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:33.165258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a995b52-d822-45b1-b61a-161cfda212b6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:33.165899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a995b52-d822-45b1-b61a-161cfda212b6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:33.166151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a995b52-d822-45b1-b61a-161cfda212b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:33.223302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 828/1666] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:33.239011 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1a995b52-d822-45b1-b61a-161cfda212b6 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:33.241130 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1a995b52-d822-45b1-b61a-161cfda212b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:33.281820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:33.457450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 839/1667] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:33.464948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49639-2156-4d69-bf02-50a13e9a096b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:33.465460 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49639-2156-4d69-bf02-50a13e9a096b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:33.465530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49639-2156-4d69-bf02-50a13e9a096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:33.540918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6bc49639-2156-4d69-bf02-50a13e9a096b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:33.546196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6bc49639-2156-4d69-bf02-50a13e9a096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:33.586610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 829/1668] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:33.590124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a995b52-d822-45b1-b61a-161cfda212b6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:33.603222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a995b52-d822-45b1-b61a-161cfda212b6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:33.618512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e464ad-964f-4b55-b268-d7c23148cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:33.620146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e464ad-964f-4b55-b268-d7c23148cf59 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:33.620442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e464ad-964f-4b55-b268-d7c23148cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:33.620730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e464ad-964f-4b55-b268-d7c23148cf59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:33.621490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e464ad-964f-4b55-b268-d7c23148cf59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:33.621804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e464ad-964f-4b55-b268-d7c23148cf59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:33.631737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e464ad-964f-4b55-b268-d7c23148cf59 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:33.665278 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 840/1669] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:33.668770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49639-2156-4d69-bf02-50a13e9a096b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:33.682610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49639-2156-4d69-bf02-50a13e9a096b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:33.695415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43636636-d549-41c3-840c-768bf0a0b324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:33.696143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43636636-d549-41c3-840c-768bf0a0b324 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:33.696430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43636636-d549-41c3-840c-768bf0a0b324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:33.696711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43636636-d549-41c3-840c-768bf0a0b324 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:33.697646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43636636-d549-41c3-840c-768bf0a0b324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:33.702663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43636636-d549-41c3-840c-768bf0a0b324 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:33.785315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 830/1670] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:33 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:33.789860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e464ad-964f-4b55-b268-d7c23148cf59 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:33.806439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e464ad-964f-4b55-b268-d7c23148cf59 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:33.947237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6df37-2f7e-49f4-9dd4-c2ba3478f65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:33.948008 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6df37-2f7e-49f4-9dd4-c2ba3478f65f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:33.948293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6df37-2f7e-49f4-9dd4-c2ba3478f65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:33.948899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6df37-2f7e-49f4-9dd4-c2ba3478f65f admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:33.949449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6df37-2f7e-49f4-9dd4-c2ba3478f65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:33.955673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6df37-2f7e-49f4-9dd4-c2ba3478f65f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:34.006425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 841/1671] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:33 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:34.010287 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43636636-d549-41c3-840c-768bf0a0b324 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:34.025478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43636636-d549-41c3-840c-768bf0a0b324 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:34.031168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6df37-2f7e-49f4-9dd4-c2ba3478f65f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:34.045506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f4d6df37-2f7e-49f4-9dd4-c2ba3478f65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:34.081878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd817fa-fba1-4051-8be7-1f82e16f78c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:34.082576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd817fa-fba1-4051-8be7-1f82e16f78c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.082867 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd817fa-fba1-4051-8be7-1f82e16f78c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.083878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd817fa-fba1-4051-8be7-1f82e16f78c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.084226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd817fa-fba1-4051-8be7-1f82e16f78c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:34.085928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6df37-2f7e-49f4-9dd4-c2ba3478f65f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:34.090692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd817fa-fba1-4051-8be7-1f82e16f78c4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:34.090937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:34.091807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.092096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.092379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] PATH_INFO: `/v3/users/da7f349193ef497aa8944224103b75cd` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.094225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7f349193ef497aa8944224103b75cd)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:34.097490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:34.124922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 831/1672] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:34.131663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd817fa-fba1-4051-8be7-1f82e16f78c4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:34.160121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd817fa-fba1-4051-8be7-1f82e16f78c4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:34.184503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976d368-d007-4f0b-ace1-694996ea8c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:34.185201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976d368-d007-4f0b-ace1-694996ea8c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.185488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976d368-d007-4f0b-ace1-694996ea8c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.185765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976d368-d007-4f0b-ace1-694996ea8c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.186509 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976d368-d007-4f0b-ace1-694996ea8c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:34.188195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976d368-d007-4f0b-ace1-694996ea8c7f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:34.207937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7f349193ef497aa8944224103b75cd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:34.210080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7f349193ef497aa8944224103b75cd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:34.231934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 832/1673] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:34.235426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976d368-d007-4f0b-ace1-694996ea8c7f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:34.237584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b976d368-d007-4f0b-ace1-694996ea8c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:34.245159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7f349193ef497aa8944224103b75cd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:34.261157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7f349193ef497aa8944224103b75cd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:34.263844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6283517-f586-4798-bd87-25b11259fdb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7f349193ef497aa8944224103b75cd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:34.274987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976d368-d007-4f0b-ace1-694996ea8c7f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:34.291377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbbc0f-050d-4983-a2bf-2edd05d7c5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:34.292562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbbc0f-050d-4983-a2bf-2edd05d7c5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.292562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbbc0f-050d-4983-a2bf-2edd05d7c5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.292876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbbc0f-050d-4983-a2bf-2edd05d7c5a1 admin admin] PATH_INFO: `/v3/projects/cf25c2b7afc54f6a8678aa540ee2817b/users/cf43ba26ac0e46a09184e075db028608/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.300448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:34.325006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 842/1674] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:34 2022] DELETE /identity/v3/users/da7f349193ef497aa8944224103b75cd => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:34.326927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061328e-c857-4df4-9ba0-2d09865b5923 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.328232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061328e-c857-4df4-9ba0-2d09865b5923 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.328523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061328e-c857-4df4-9ba0-2d09865b5923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.336952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbbc0f-050d-4983-a2bf-2edd05d7c5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf25c2b7afc54f6a8678aa540ee2817b, user_id=cf43ba26ac0e46a09184e075db028608, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:34.339702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbbc0f-050d-4983-a2bf-2edd05d7c5a1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:34.362772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:34.373045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 833/1675] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:34 2022] PUT /identity/v3/projects/cf25c2b7afc54f6a8678aa540ee2817b/users/cf43ba26ac0e46a09184e075db028608/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:34.374707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5b887-b260-4190-ab0d-f21474b7464c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.375004 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5b887-b260-4190-ab0d-f21474b7464c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.375288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5b887-b260-4190-ab0d-f21474b7464c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.408342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5061328e-c857-4df4-9ba0-2d09865b5923 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:34.419054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5061328e-c857-4df4-9ba0-2d09865b5923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:34.453211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bbe5b887-b260-4190-ab0d-f21474b7464c None None] MFA Rules not processed for user `cf43ba26ac0e46a09184e075db028608`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:34.454469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bbe5b887-b260-4190-ab0d-f21474b7464c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:34.609174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 834/1676] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:34.614303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5b887-b260-4190-ab0d-f21474b7464c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:34.623650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 843/1677] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:34.631830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5b887-b260-4190-ab0d-f21474b7464c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:34.647254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdaea34-c724-4d53-b90a-7052f2f02c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:34.648362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdaea34-c724-4d53-b90a-7052f2f02c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.648863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdaea34-c724-4d53-b90a-7052f2f02c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.649349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdaea34-c724-4d53-b90a-7052f2f02c5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.650274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdaea34-c724-4d53-b90a-7052f2f02c5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:34.652493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdaea34-c724-4d53-b90a-7052f2f02c5b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:34.703008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 835/1678] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:34 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:34.707205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061328e-c857-4df4-9ba0-2d09865b5923 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:34.718932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdaea34-c724-4d53-b90a-7052f2f02c5b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:34.720747 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061328e-c857-4df4-9ba0-2d09865b5923 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:34.721341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6bdaea34-c724-4d53-b90a-7052f2f02c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:34.836650 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdaea34-c724-4d53-b90a-7052f2f02c5b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:34.854093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff6109-67a1-430f-9127-0ddd05149b4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:34.855823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff6109-67a1-430f-9127-0ddd05149b4d service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.856312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff6109-67a1-430f-9127-0ddd05149b4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.856799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff6109-67a1-430f-9127-0ddd05149b4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.882733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d056c-c085-495c-90e1-3f5755eaa214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:34.884183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d056c-c085-495c-90e1-3f5755eaa214 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.884474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d056c-c085-495c-90e1-3f5755eaa214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.889508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d056c-c085-495c-90e1-3f5755eaa214 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.890272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d056c-c085-495c-90e1-3f5755eaa214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:34.892282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d056c-c085-495c-90e1-3f5755eaa214 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:34.946626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 844/1679] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:34 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:34.947772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d056c-c085-495c-90e1-3f5755eaa214 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:34.978120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d056c-c085-495c-90e1-3f5755eaa214 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:34.983104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92351c02-64ab-4a89-8573-0f27eabe6680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:34.983773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92351c02-64ab-4a89-8573-0f27eabe6680 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:34.984064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92351c02-64ab-4a89-8573-0f27eabe6680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:34.984344 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92351c02-64ab-4a89-8573-0f27eabe6680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:34.985093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92351c02-64ab-4a89-8573-0f27eabe6680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:34.994637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92351c02-64ab-4a89-8573-0f27eabe6680 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:34.997008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff6109-67a1-430f-9127-0ddd05149b4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:34.998771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff6109-67a1-430f-9127-0ddd05149b4d service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:35.020077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 845/1680] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:35.024375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92351c02-64ab-4a89-8573-0f27eabe6680 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:35.028616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 836/1681] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:41:35.035707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92351c02-64ab-4a89-8573-0f27eabe6680 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:35.041946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde6b65-3874-4a4f-866c-d718bfa4113e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:35.042870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde6b65-3874-4a4f-866c-d718bfa4113e admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:35.043644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde6b65-3874-4a4f-866c-d718bfa4113e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:35.043935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde6b65-3874-4a4f-866c-d718bfa4113e admin admin] PATH_INFO: `/v3/projects/d6044e66e27f4b788c77eb76095ad3b8/users/3b939ecbbbba4d96a419cf1809e7339b/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:35.077834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde6b65-3874-4a4f-866c-d718bfa4113e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6044e66e27f4b788c77eb76095ad3b8, user_id=3b939ecbbbba4d96a419cf1809e7339b, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:35.081275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde6b65-3874-4a4f-866c-d718bfa4113e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:35.110293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 846/1682] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:35 2022] PUT /identity/v3/projects/d6044e66e27f4b788c77eb76095ad3b8/users/3b939ecbbbba4d96a419cf1809e7339b/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:35.111765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff6109-67a1-430f-9127-0ddd05149b4d service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:35.128906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:35.142046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff6109-67a1-430f-9127-0ddd05149b4d service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:35.308041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d31efd-dc42-42b3-949d-80dc7af24c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:35.308884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d31efd-dc42-42b3-949d-80dc7af24c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:35.309177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d31efd-dc42-42b3-949d-80dc7af24c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:35.309519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d31efd-dc42-42b3-949d-80dc7af24c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:35.310264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d31efd-dc42-42b3-949d-80dc7af24c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:35.316283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d31efd-dc42-42b3-949d-80dc7af24c9e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:35.345183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 837/1683] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:35.346296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde6b65-3874-4a4f-866c-d718bfa4113e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:35.377371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde6b65-3874-4a4f-866c-d718bfa4113e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:35.397107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6b706-c5aa-40e4-b1af-cf8b64508c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:35.400174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6b706-c5aa-40e4-b1af-cf8b64508c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:35.400470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6b706-c5aa-40e4-b1af-cf8b64508c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:35.401570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6b706-c5aa-40e4-b1af-cf8b64508c13 admin admin] PATH_INFO: `/v3/projects/d6044e66e27f4b788c77eb76095ad3b8/users/3b939ecbbbba4d96a419cf1809e7339b/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:35.408689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6b706-c5aa-40e4-b1af-cf8b64508c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6044e66e27f4b788c77eb76095ad3b8, user_id=3b939ecbbbba4d96a419cf1809e7339b, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:35.411551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6b706-c5aa-40e4-b1af-cf8b64508c13 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:35.441629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 847/1684] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:35 2022] PUT /identity/v3/projects/d6044e66e27f4b788c77eb76095ad3b8/users/3b939ecbbbba4d96a419cf1809e7339b/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:35.456634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df269926-76b7-40cf-ad90-3bd006af896e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:35.456953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df269926-76b7-40cf-ad90-3bd006af896e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:35.457259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df269926-76b7-40cf-ad90-3bd006af896e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:35.520869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-df269926-76b7-40cf-ad90-3bd006af896e None None] MFA Rules not processed for user `3b939ecbbbba4d96a419cf1809e7339b`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:35.522318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-df269926-76b7-40cf-ad90-3bd006af896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:35.697456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 838/1685] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:35.709098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6b706-c5aa-40e4-b1af-cf8b64508c13 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:35.711156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cfb6b706-c5aa-40e4-b1af-cf8b64508c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:35.788849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:35.798495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6b706-c5aa-40e4-b1af-cf8b64508c13 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:35.807057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba990c4a-b7c9-4d8b-8e1b-d11ef93021a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:35.809812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba990c4a-b7c9-4d8b-8e1b-d11ef93021a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:35.810208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba990c4a-b7c9-4d8b-8e1b-d11ef93021a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:35.810589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba990c4a-b7c9-4d8b-8e1b-d11ef93021a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:35.884417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cf2a1-a310-4cad-979f-5884aa1cb9c1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:35.886216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cf2a1-a310-4cad-979f-5884aa1cb9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:35.886596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cf2a1-a310-4cad-979f-5884aa1cb9c1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:35.888364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 839/1686] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 13 20:41:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:41:35.892299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba990c4a-b7c9-4d8b-8e1b-d11ef93021a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:35.894756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba990c4a-b7c9-4d8b-8e1b-d11ef93021a8 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:35.899329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73049b7a-75b0-4636-92a0-9b2a02d86851 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:35.900768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73049b7a-75b0-4636-92a0-9b2a02d86851 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:35.901158 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73049b7a-75b0-4636-92a0-9b2a02d86851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:35.948878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 848/1687] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:35.992664 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-73049b7a-75b0-4636-92a0-9b2a02d86851 None None] MFA Rules not processed for user `c57d44c19d15496e96c6e37510ce67fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:36.010474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-73049b7a-75b0-4636-92a0-9b2a02d86851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:36.196793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 840/1688] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:41:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 13 20:41:36.198862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba990c4a-b7c9-4d8b-8e1b-d11ef93021a8 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:36.216333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba990c4a-b7c9-4d8b-8e1b-d11ef93021a8 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:36.240705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ec596-4c2a-4427-8cea-46c9ecce0745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:36.241047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ec596-4c2a-4427-8cea-46c9ecce0745 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:36.241308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ec596-4c2a-4427-8cea-46c9ecce0745 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:36.241600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ec596-4c2a-4427-8cea-46c9ecce0745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:36.242539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73049b7a-75b0-4636-92a0-9b2a02d86851 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:36.257250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73049b7a-75b0-4636-92a0-9b2a02d86851 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:36.262044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f36877-f887-4efb-b816-8a5518a28623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:36.263442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f36877-f887-4efb-b816-8a5518a28623 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:36.267075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f36877-f887-4efb-b816-8a5518a28623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:36.267799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f36877-f887-4efb-b816-8a5518a28623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:36.279719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f36877-f887-4efb-b816-8a5518a28623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:36.281694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f36877-f887-4efb-b816-8a5518a28623 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:36.293272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 841/1689] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:41:36.405848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f36877-f887-4efb-b816-8a5518a28623 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:36.408023 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c2f36877-f887-4efb-b816-8a5518a28623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:36.430466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:36.543232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ec596-4c2a-4427-8cea-46c9ecce0745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:36.559523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ec596-4c2a-4427-8cea-46c9ecce0745 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:36.588510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f36877-f887-4efb-b816-8a5518a28623 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:36.599296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 849/1690] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:36.601209 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:36.602706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:36.603544 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:36.604100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] PATH_INFO: `/v3/projects/466c279cb08c4889a3fb42d913b46cfe` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:36.605719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] RBAC: Authorizing `identity:delete_project(project_id=466c279cb08c4889a3fb42d913b46cfe)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:36.616955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:36.728799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466c279cb08c4889a3fb42d913b46cfe', 'request_id': 'req-2dcf2f58-4354-43b8-a645-f99af8e0478e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:36.743655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466c279cb08c4889a3fb42d913b46cfe', 'request_id': 'req-2dcf2f58-4354-43b8-a645-f99af8e0478e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:36.753673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466c279cb08c4889a3fb42d913b46cfe', 'request_id': 'req-2dcf2f58-4354-43b8-a645-f99af8e0478e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:36.782990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] The token cache is being invalidate because project 466c279cb08c4889a3fb42d913b46cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:36.783515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:36.784271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dcf2f58-4354-43b8-a645-f99af8e0478e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:36.786787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 842/1691] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:36 2022] DELETE /identity/v3/projects/466c279cb08c4889a3fb42d913b46cfe => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:36.790705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ec596-4c2a-4427-8cea-46c9ecce0745 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:36.803504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ec596-4c2a-4427-8cea-46c9ecce0745 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:36.936499 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:36.937642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:36.939161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:36.941261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] PATH_INFO: `/v3/users/cec6cb434bb7493dbc2dbe368785fabe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:36.945304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec6cb434bb7493dbc2dbe368785fabe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:36.951196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:37.062909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec6cb434bb7493dbc2dbe368785fabe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:37.063713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec6cb434bb7493dbc2dbe368785fabe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:37.107165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec6cb434bb7493dbc2dbe368785fabe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:37.117398 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec6cb434bb7493dbc2dbe368785fabe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:37.120034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be90ba93-5990-47fa-b143-ef898009e2fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec6cb434bb7493dbc2dbe368785fabe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:37.148229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 850/1692] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:36 2022] DELETE /identity/v3/users/cec6cb434bb7493dbc2dbe368785fabe => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:38.149733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8823d-14b1-4cb2-804b-f98641c1d885 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:38.150296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8823d-14b1-4cb2-804b-f98641c1d885 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:38.150296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8823d-14b1-4cb2-804b-f98641c1d885 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:38.151684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 843/1693] 10.222.0.47 () {54 vars in 960 bytes} [Wed Jul 13 20:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:41:38.157576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb002-36ea-4d56-8177-8d74ff4499a5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:38.158144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb002-36ea-4d56-8177-8d74ff4499a5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:38.158252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb002-36ea-4d56-8177-8d74ff4499a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:38.183760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:38.194872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-814cb002-36ea-4d56-8177-8d74ff4499a5 None None] MFA Rules not processed for user `cbafea9ed6494a3f9aeba4140a32de4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:38.197568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-814cb002-36ea-4d56-8177-8d74ff4499a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:38.357877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 851/1694] 10.222.0.47 () {58 vars in 1057 bytes} [Wed Jul 13 20:41:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:38.384309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8823d-14b1-4cb2-804b-f98641c1d885 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:38.413547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8823d-14b1-4cb2-804b-f98641c1d885 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:38.434275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee53051-6ed1-4248-a407-f7f503780bcd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:38.435428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee53051-6ed1-4248-a407-f7f503780bcd service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:38.437338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee53051-6ed1-4248-a407-f7f503780bcd service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:38.439584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee53051-6ed1-4248-a407-f7f503780bcd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:38.448111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-aee53051-6ed1-4248-a407-f7f503780bcd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:38.475386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee53051-6ed1-4248-a407-f7f503780bcd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:38.480612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee53051-6ed1-4248-a407-f7f503780bcd service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:38.506264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 844/1695] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:38.713726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb002-36ea-4d56-8177-8d74ff4499a5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:38.718324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-814cb002-36ea-4d56-8177-8d74ff4499a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:38.927211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb002-36ea-4d56-8177-8d74ff4499a5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:38.938510 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:38.945438 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:38.945854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:38.946148 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] PATH_INFO: `/v3/projects/854b3e18898941eb93d1bf64e9f41135` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:38.949283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=854b3e18898941eb93d1bf64e9f41135)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:38.962173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:39.140361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854b3e18898941eb93d1bf64e9f41135', 'request_id': 'req-c371c118-c1fd-4029-8d87-349bb8180b56'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.194352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854b3e18898941eb93d1bf64e9f41135', 'request_id': 'req-c371c118-c1fd-4029-8d87-349bb8180b56'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.227146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854b3e18898941eb93d1bf64e9f41135', 'request_id': 'req-c371c118-c1fd-4029-8d87-349bb8180b56'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.317411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] The token cache is being invalidate because project 854b3e18898941eb93d1bf64e9f41135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:39.317411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.320262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.321847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 852/1696] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:38 2022] DELETE /identity/v3/projects/854b3e18898941eb93d1bf64e9f41135 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:39.327357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2d6c3-3c2d-4836-8d0a-e682c79c3b76 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:39.327814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2d6c3-3c2d-4836-8d0a-e682c79c3b76 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:39.328047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2d6c3-3c2d-4836-8d0a-e682c79c3b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:39.356321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1dd2d6c3-3c2d-4836-8d0a-e682c79c3b76 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:39.358122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1dd2d6c3-3c2d-4836-8d0a-e682c79c3b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:39.404123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:39.405648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:39.499292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:39.530225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 845/1697] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:39.531807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f27fd-d2bd-484e-8e6e-972aa4b8e8c5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:39.532102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f27fd-d2bd-484e-8e6e-972aa4b8e8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:39.532386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f27fd-d2bd-484e-8e6e-972aa4b8e8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:39.581450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371c118-c1fd-4029-8d87-349bb8180b56 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:39.592873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-963f27fd-d2bd-484e-8e6e-972aa4b8e8c5 None None] MFA Rules not processed for user `0b22e96941dd414da4fa22ed4bd36984`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:39.595152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-963f27fd-d2bd-484e-8e6e-972aa4b8e8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:39.595298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:39.596628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:39.597871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:39.599363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] PATH_INFO: `/v3/users/c501417991064faa85556c0f0651e99f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:39.601756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c501417991064faa85556c0f0651e99f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:39.610150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:39.618377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:39.697675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c501417991064faa85556c0f0651e99f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.698784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c501417991064faa85556c0f0651e99f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.717784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c501417991064faa85556c0f0651e99f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.730937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c501417991064faa85556c0f0651e99f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.731821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ea71aa8-be62-44da-a047-bfa1cc7a2f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c501417991064faa85556c0f0651e99f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:39.752470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 853/1698] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:39 2022] DELETE /identity/v3/users/c501417991064faa85556c0f0651e99f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:39.754958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1211c-a14b-4db5-bf40-fc12aeabfe85 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:39.755574 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1211c-a14b-4db5-bf40-fc12aeabfe85 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:39.756099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1211c-a14b-4db5-bf40-fc12aeabfe85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:39.757799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 846/1699] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:39.759301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813217-aa5c-4821-bd28-5d006ee93ac5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:39.759622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813217-aa5c-4821-bd28-5d006ee93ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:39.759904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813217-aa5c-4821-bd28-5d006ee93ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:39.784753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-23f1211c-a14b-4db5-bf40-fc12aeabfe85 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:39.786900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-23f1211c-a14b-4db5-bf40-fc12aeabfe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:39.799312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-48813217-aa5c-4821-bd28-5d006ee93ac5 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:39.801803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-48813217-aa5c-4821-bd28-5d006ee93ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:39.928032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 854/1700] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:39.937108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1211c-a14b-4db5-bf40-fc12aeabfe85 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:39.942414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-23f1211c-a14b-4db5-bf40-fc12aeabfe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:39.971782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 847/1701] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:39.974566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813217-aa5c-4821-bd28-5d006ee93ac5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:39.998789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813217-aa5c-4821-bd28-5d006ee93ac5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:40.007211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:40.008361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:40.008879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:40.009419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:40.016329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:40.019443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:40.022030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:40.035021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:40.056341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 848/1702] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:40.058633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:40.063759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:40.095237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1211c-a14b-4db5-bf40-fc12aeabfe85 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:40.113585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723af706-739b-4a9b-b7fd-ee55c0cb79dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:40.116230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af706-739b-4a9b-b7fd-ee55c0cb79dc service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:40.117742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af706-739b-4a9b-b7fd-ee55c0cb79dc service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:40.118314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af706-739b-4a9b-b7fd-ee55c0cb79dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:40.168783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a621696-3a89-41c4-850b-18d2500c5938 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:40.178550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb96459-d857-4c87-a1d1-d1549885fdd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:40.179245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb96459-d857-4c87-a1d1-d1549885fdd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:40.179761 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb96459-d857-4c87-a1d1-d1549885fdd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:40.180112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb96459-d857-4c87-a1d1-d1549885fdd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:40.193411 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb96459-d857-4c87-a1d1-d1549885fdd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:40.195082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb96459-d857-4c87-a1d1-d1549885fdd7 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:40.227537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 849/1703] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:40.228799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb96459-d857-4c87-a1d1-d1549885fdd7 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:40.265459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb96459-d857-4c87-a1d1-d1549885fdd7 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:40.278822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723af706-739b-4a9b-b7fd-ee55c0cb79dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:40.281644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723af706-739b-4a9b-b7fd-ee55c0cb79dc service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:40.285531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aef804-e989-4edc-a47f-7de5be0472e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:40.295065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aef804-e989-4edc-a47f-7de5be0472e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:40.295355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aef804-e989-4edc-a47f-7de5be0472e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:40.295659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aef804-e989-4edc-a47f-7de5be0472e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:40.300039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aef804-e989-4edc-a47f-7de5be0472e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:40.308023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 855/1704] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:40.309174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723af706-739b-4a9b-b7fd-ee55c0cb79dc service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:40.314045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-723af706-739b-4a9b-b7fd-ee55c0cb79dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:40.315665 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aef804-e989-4edc-a47f-7de5be0472e8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:40.346918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723af706-739b-4a9b-b7fd-ee55c0cb79dc service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:40.354391 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:40.355314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:40.355647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:40.355922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] PATH_INFO: `/v3/users/cf43ba26ac0e46a09184e075db028608` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:40.359363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf43ba26ac0e46a09184e075db028608)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:40.361430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:40.389957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 850/1705] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:40 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:40.395093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aef804-e989-4edc-a47f-7de5be0472e8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:40.467591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:40.526577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf43ba26ac0e46a09184e075db028608'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:40.528027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf43ba26ac0e46a09184e075db028608'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:40.574984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf43ba26ac0e46a09184e075db028608'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:40.584691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aef804-e989-4edc-a47f-7de5be0472e8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:40.602066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f9311-26d2-4702-9fa6-48f8d1e471b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:40.602765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f9311-26d2-4702-9fa6-48f8d1e471b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:40.603049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f9311-26d2-4702-9fa6-48f8d1e471b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:40.603321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f9311-26d2-4702-9fa6-48f8d1e471b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:40.604053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f9311-26d2-4702-9fa6-48f8d1e471b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:40.606004 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f9311-26d2-4702-9fa6-48f8d1e471b3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:40.616649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf43ba26ac0e46a09184e075db028608'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:40.617406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9a61f16a-d0fc-44a0-83e7-c33e0715040b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf43ba26ac0e46a09184e075db028608'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:40.653262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 856/1706] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:40 2022] DELETE /identity/v3/users/cf43ba26ac0e46a09184e075db028608 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:40.655784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724963cb-1762-4ff2-b8e7-6ee889f85681 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:40.656082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724963cb-1762-4ff2-b8e7-6ee889f85681 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:40.656364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724963cb-1762-4ff2-b8e7-6ee889f85681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:40.681784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 851/1707] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:40 2022] POST /identity/v3/users => generated 285 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:40.688654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f9311-26d2-4702-9fa6-48f8d1e471b3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:40.691111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a71f9311-26d2-4702-9fa6-48f8d1e471b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:40.725284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-724963cb-1762-4ff2-b8e7-6ee889f85681 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:40.726711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-724963cb-1762-4ff2-b8e7-6ee889f85681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:40.738642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f9311-26d2-4702-9fa6-48f8d1e471b3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:40.943501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 857/1708] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:40.955258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724963cb-1762-4ff2-b8e7-6ee889f85681 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:40.957091 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-724963cb-1762-4ff2-b8e7-6ee889f85681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:40.958424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94619a63-049d-480b-ad78-c248ff33f857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:40.959242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94619a63-049d-480b-ad78-c248ff33f857 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:40.959552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94619a63-049d-480b-ad78-c248ff33f857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:40.959836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94619a63-049d-480b-ad78-c248ff33f857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:40.960916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94619a63-049d-480b-ad78-c248ff33f857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:40.962431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94619a63-049d-480b-ad78-c248ff33f857 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:40.975075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:41.005274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 852/1709] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:41.011245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94619a63-049d-480b-ad78-c248ff33f857 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:41.026082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94619a63-049d-480b-ad78-c248ff33f857 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:41.032156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb255b-979e-48de-8e89-2fb3b61e6b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:41.032851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb255b-979e-48de-8e89-2fb3b61e6b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:41.033297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb255b-979e-48de-8e89-2fb3b61e6b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:41.033580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb255b-979e-48de-8e89-2fb3b61e6b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:41.034317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb255b-979e-48de-8e89-2fb3b61e6b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:41.036581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb255b-979e-48de-8e89-2fb3b61e6b9e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:41.053257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 853/1710] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:41.054199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb255b-979e-48de-8e89-2fb3b61e6b9e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:41.075788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb255b-979e-48de-8e89-2fb3b61e6b9e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:41.081919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df2ee9-b362-481a-860e-0a2f8e8f4168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:41.082838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2ee9-b362-481a-860e-0a2f8e8f4168 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:41.083299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2ee9-b362-481a-860e-0a2f8e8f4168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:41.083636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2ee9-b362-481a-860e-0a2f8e8f4168 admin admin] PATH_INFO: `/v3/projects/557486c3564a40bc8476717f908d3b48/users/7705af78464944038f8017435a7401ae/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:41.103438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df2ee9-b362-481a-860e-0a2f8e8f4168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557486c3564a40bc8476717f908d3b48, user_id=7705af78464944038f8017435a7401ae, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:41.106337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df2ee9-b362-481a-860e-0a2f8e8f4168 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:41.128777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724963cb-1762-4ff2-b8e7-6ee889f85681 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:41.132759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 854/1711] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:41 2022] PUT /identity/v3/projects/557486c3564a40bc8476717f908d3b48/users/7705af78464944038f8017435a7401ae/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:41.144614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4b186-488d-45f2-903f-b0f967cb92e2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:41.145193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4b186-488d-45f2-903f-b0f967cb92e2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:41.146242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4b186-488d-45f2-903f-b0f967cb92e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:41.206421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6be4b186-488d-45f2-903f-b0f967cb92e2 None None] MFA Rules not processed for user `7705af78464944038f8017435a7401ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:41.209838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6be4b186-488d-45f2-903f-b0f967cb92e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:41.274214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c464fa-f7b1-42bd-a0bc-6784da17d66b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:41.274899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c464fa-f7b1-42bd-a0bc-6784da17d66b service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:41.277377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c464fa-f7b1-42bd-a0bc-6784da17d66b service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:41.277678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c464fa-f7b1-42bd-a0bc-6784da17d66b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:41.392048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 855/1712] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:41.393323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4b186-488d-45f2-903f-b0f967cb92e2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:41.404195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:41.455172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c464fa-f7b1-42bd-a0bc-6784da17d66b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:41.456928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c464fa-f7b1-42bd-a0bc-6784da17d66b service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:41.487246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4b186-488d-45f2-903f-b0f967cb92e2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:41.497924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662cfd5-6a69-4fbe-bc67-b594e2c41405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:41.501505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662cfd5-6a69-4fbe-bc67-b594e2c41405 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:41.502050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662cfd5-6a69-4fbe-bc67-b594e2c41405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:41.502343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662cfd5-6a69-4fbe-bc67-b594e2c41405 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:41.505191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662cfd5-6a69-4fbe-bc67-b594e2c41405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:41.505843 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 858/1713] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:41.510190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662cfd5-6a69-4fbe-bc67-b594e2c41405 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:41.604569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 856/1714] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:41 2022] POST /identity/v3/projects => generated 369 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:41.608285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c464fa-f7b1-42bd-a0bc-6784da17d66b service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:41.627378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c464fa-f7b1-42bd-a0bc-6784da17d66b service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:41.850907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba27e9-adcf-47c9-b4a6-3bcd49a6dc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:41.851649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba27e9-adcf-47c9-b4a6-3bcd49a6dc53 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:41.852073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba27e9-adcf-47c9-b4a6-3bcd49a6dc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:41.852351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba27e9-adcf-47c9-b4a6-3bcd49a6dc53 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:41.853077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba27e9-adcf-47c9-b4a6-3bcd49a6dc53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:41.857469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba27e9-adcf-47c9-b4a6-3bcd49a6dc53 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:41.895980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:41.909827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 859/1715] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:41 2022] POST /identity/v3/users => generated 291 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:41.913404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662cfd5-6a69-4fbe-bc67-b594e2c41405 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:41.935769 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662cfd5-6a69-4fbe-bc67-b594e2c41405 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:41.948195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e71e-2d8b-421c-8b32-56eefb5864ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:41.949627 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e71e-2d8b-421c-8b32-56eefb5864ad admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:41.950089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e71e-2d8b-421c-8b32-56eefb5864ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:41.952779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e71e-2d8b-421c-8b32-56eefb5864ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:41.957429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950e71e-2d8b-421c-8b32-56eefb5864ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:41.959870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950e71e-2d8b-421c-8b32-56eefb5864ad admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:41.986826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 857/1716] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:41.989580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba27e9-adcf-47c9-b4a6-3bcd49a6dc53 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:42.017842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba27e9-adcf-47c9-b4a6-3bcd49a6dc53 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:42.034833 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294daa0-29dd-423d-8a59-67c2b0aef73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:42.035796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294daa0-29dd-423d-8a59-67c2b0aef73a admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:42.036084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294daa0-29dd-423d-8a59-67c2b0aef73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:42.044059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294daa0-29dd-423d-8a59-67c2b0aef73a admin admin] PATH_INFO: `/v3/projects/269b18b56dc645c8b606ab6e78f3e5ce/users/2c13d340b2fe416db7948576b6441b99/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:42.109691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294daa0-29dd-423d-8a59-67c2b0aef73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=269b18b56dc645c8b606ab6e78f3e5ce, user_id=2c13d340b2fe416db7948576b6441b99, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:42.120829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294daa0-29dd-423d-8a59-67c2b0aef73a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:42.172269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 860/1717] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:41 2022] PUT /identity/v3/projects/269b18b56dc645c8b606ab6e78f3e5ce/users/2c13d340b2fe416db7948576b6441b99/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:42.173872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e71e-2d8b-421c-8b32-56eefb5864ad admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:42.188628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e71e-2d8b-421c-8b32-56eefb5864ad admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:42.267091 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294daa0-29dd-423d-8a59-67c2b0aef73a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:42.269847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a294daa0-29dd-423d-8a59-67c2b0aef73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:42.397029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d972-a297-4bf1-a876-0b366a020d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:42.398839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d972-a297-4bf1-a876-0b366a020d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:42.399184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d972-a297-4bf1-a876-0b366a020d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:42.400592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d972-a297-4bf1-a876-0b366a020d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:42.403080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8d972-a297-4bf1-a876-0b366a020d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:42.416219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8d972-a297-4bf1-a876-0b366a020d1b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:42.456295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 858/1718] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:42.460669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d972-a297-4bf1-a876-0b366a020d1b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:42.488804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d972-a297-4bf1-a876-0b366a020d1b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:42.501963 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294daa0-29dd-423d-8a59-67c2b0aef73a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:42.524061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6298-5bc3-4e9c-a40b-7e2edb0b6d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:42.524957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:42.525866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:42.526157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:42.526425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6298-5bc3-4e9c-a40b-7e2edb0b6d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:42.526542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] PATH_INFO: `/v3/projects/cf25c2b7afc54f6a8678aa540ee2817b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:42.531271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6298-5bc3-4e9c-a40b-7e2edb0b6d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:42.531721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6298-5bc3-4e9c-a40b-7e2edb0b6d43 admin admin] PATH_INFO: `/v3/projects/269b18b56dc645c8b606ab6e78f3e5ce/users/2c13d340b2fe416db7948576b6441b99/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:42.536200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf25c2b7afc54f6a8678aa540ee2817b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:42.538295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:42.546175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6298-5bc3-4e9c-a40b-7e2edb0b6d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=269b18b56dc645c8b606ab6e78f3e5ce, user_id=2c13d340b2fe416db7948576b6441b99, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:42.555421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6298-5bc3-4e9c-a40b-7e2edb0b6d43 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:42.604255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 859/1719] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:42 2022] PUT /identity/v3/projects/269b18b56dc645c8b606ab6e78f3e5ce/users/2c13d340b2fe416db7948576b6441b99/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:42.606381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c690b5-66f6-4bd9-a1ce-3e95cbfc30c6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:42.606945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c690b5-66f6-4bd9-a1ce-3e95cbfc30c6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:42.607611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c690b5-66f6-4bd9-a1ce-3e95cbfc30c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:42.707219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-51c690b5-66f6-4bd9-a1ce-3e95cbfc30c6 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:42.708157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf25c2b7afc54f6a8678aa540ee2817b', 'request_id': 'req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:42.708973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-51c690b5-66f6-4bd9-a1ce-3e95cbfc30c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:42.727320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf25c2b7afc54f6a8678aa540ee2817b', 'request_id': 'req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:42.744887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf25c2b7afc54f6a8678aa540ee2817b', 'request_id': 'req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:42.795764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] The token cache is being invalidate because project cf25c2b7afc54f6a8678aa540ee2817b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:42.796094 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:42.796811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6d536bf3-f2fa-4d7a-8f0f-03c435317c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:42.797986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:42.798435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 861/1720] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:42 2022] DELETE /identity/v3/projects/cf25c2b7afc54f6a8678aa540ee2817b => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:42.810176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7212a-018d-45ed-8478-9317c0ae7560 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:42.810507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7212a-018d-45ed-8478-9317c0ae7560 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:42.810746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7212a-018d-45ed-8478-9317c0ae7560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:42.895486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5ff7212a-018d-45ed-8478-9317c0ae7560 None None] MFA Rules not processed for user `2c13d340b2fe416db7948576b6441b99`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:42.895486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5ff7212a-018d-45ed-8478-9317c0ae7560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:42.946728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 860/1721] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:41:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:42.947673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c690b5-66f6-4bd9-a1ce-3e95cbfc30c6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:42.950230 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-51c690b5-66f6-4bd9-a1ce-3e95cbfc30c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:43.059688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:43.096135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 862/1722] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:42 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:43.097373 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7212a-018d-45ed-8478-9317c0ae7560 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:43.102519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5ff7212a-018d-45ed-8478-9317c0ae7560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:43.118913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7212a-018d-45ed-8478-9317c0ae7560 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:43.126397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:43.127841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:43.127927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c690b5-66f6-4bd9-a1ce-3e95cbfc30c6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:43.128411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:43.129128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] PATH_INFO: `/v3/projects/84d603577ad942e2bfd7b14995dfc5ec` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:43.130524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84d603577ad942e2bfd7b14995dfc5ec)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:43.132948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:43.141037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:43.142032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:43.142473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:43.142823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] PATH_INFO: `/v3/users/3b939ecbbbba4d96a419cf1809e7339b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:43.152381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b939ecbbbba4d96a419cf1809e7339b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:43.154495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:43.225499 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84d603577ad942e2bfd7b14995dfc5ec', 'request_id': 'req-63bd89f5-76c0-4be2-9c66-50093ff25b03'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.237022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84d603577ad942e2bfd7b14995dfc5ec', 'request_id': 'req-63bd89f5-76c0-4be2-9c66-50093ff25b03'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.270666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84d603577ad942e2bfd7b14995dfc5ec', 'request_id': 'req-63bd89f5-76c0-4be2-9c66-50093ff25b03'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.286292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b939ecbbbba4d96a419cf1809e7339b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.287137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b939ecbbbba4d96a419cf1809e7339b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.309660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] The token cache is being invalidate because project 84d603577ad942e2bfd7b14995dfc5ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:43.311357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.312944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.325176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 863/1723] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:43 2022] DELETE /identity/v3/projects/84d603577ad942e2bfd7b14995dfc5ec => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:43.326295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:43.329292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:43.337830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b939ecbbbba4d96a419cf1809e7339b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.362853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b939ecbbbba4d96a419cf1809e7339b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.389447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b939ecbbbba4d96a419cf1809e7339b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:43.401890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 861/1724] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:42 2022] DELETE /identity/v3/users/3b939ecbbbba4d96a419cf1809e7339b => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:43.402935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:43.405229 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:43.480978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:43.482001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd89f5-76c0-4be2-9c66-50093ff25b03 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:43.567261 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a08087-5237-46f7-a4f1-411c81993399 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:43.579955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:43.580731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:43.581225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:43.581547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:43.584267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:43.602408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a65e0b-902a-41fc-92e8-06d72503ad16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:43.603045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65e0b-902a-41fc-92e8-06d72503ad16 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:43.603330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65e0b-902a-41fc-92e8-06d72503ad16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:43.603631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65e0b-902a-41fc-92e8-06d72503ad16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:43.609591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-34a65e0b-902a-41fc-92e8-06d72503ad16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:43.692456 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:43.741581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a65e0b-902a-41fc-92e8-06d72503ad16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:43.750621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a65e0b-902a-41fc-92e8-06d72503ad16 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:43.763830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:43.767620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:43.779284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 864/1725] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:43 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:43.780758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cc5c7-7ab1-4cc3-baa3-c8998c8182f5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:43.789189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cc5c7-7ab1-4cc3-baa3-c8998c8182f5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:43.789189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cc5c7-7ab1-4cc3-baa3-c8998c8182f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:43.818795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 862/1726] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:41:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:41:43.884848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9a8cc5c7-7ab1-4cc3-baa3-c8998c8182f5 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:43.889580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9a8cc5c7-7ab1-4cc3-baa3-c8998c8182f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:43.897636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:43.899439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:43.919254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b9e7-c5e1-4466-81e4-d23e9ccea53d service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:43.929515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a3e10-3457-45ed-9f78-d73f4a4384e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:43.930547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a3e10-3457-45ed-9f78-d73f4a4384e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:43.931056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a3e10-3457-45ed-9f78-d73f4a4384e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:43.931731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a3e10-3457-45ed-9f78-d73f4a4384e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:43.934932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-da0a3e10-3457-45ed-9f78-d73f4a4384e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:44.118513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 865/1727] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:44.134102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b04d0f-d541-42b6-b5a9-f0703ad653c9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:44.139054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b04d0f-d541-42b6-b5a9-f0703ad653c9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:44.139610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b04d0f-d541-42b6-b5a9-f0703ad653c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:44.166436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a3e10-3457-45ed-9f78-d73f4a4384e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:44.172492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a3e10-3457-45ed-9f78-d73f4a4384e6 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:44.194731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b3b04d0f-d541-42b6-b5a9-f0703ad653c9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:44.196119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b3b04d0f-d541-42b6-b5a9-f0703ad653c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:44.198343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 863/1728] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:44.243418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 866/1729] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:44.246856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a3e10-3457-45ed-9f78-d73f4a4384e6 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:44.262377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a3e10-3457-45ed-9f78-d73f4a4384e6 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:44.273291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b3194-1c9e-430f-98d9-f39c9e6262a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:44.279596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b3194-1c9e-430f-98d9-f39c9e6262a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:44.280166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b3194-1c9e-430f-98d9-f39c9e6262a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:44.281614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b3194-1c9e-430f-98d9-f39c9e6262a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:44.285226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b3194-1c9e-430f-98d9-f39c9e6262a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:44.291609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b3194-1c9e-430f-98d9-f39c9e6262a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:44.293407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b3194-1c9e-430f-98d9-f39c9e6262a5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:44.359749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b04d0f-d541-42b6-b5a9-f0703ad653c9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:44.363520 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 864/1730] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:44.377594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b04d0f-d541-42b6-b5a9-f0703ad653c9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:44.394220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df39b2a-74e3-41df-a68a-50e3d28088ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:44.397238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df39b2a-74e3-41df-a68a-50e3d28088ad admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:44.397632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df39b2a-74e3-41df-a68a-50e3d28088ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:44.397834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df39b2a-74e3-41df-a68a-50e3d28088ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:44.400874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df39b2a-74e3-41df-a68a-50e3d28088ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:44.407038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df39b2a-74e3-41df-a68a-50e3d28088ad admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:44.489861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 867/1731] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:44 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:41:44.493876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b3194-1c9e-430f-98d9-f39c9e6262a5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:44.507106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b3194-1c9e-430f-98d9-f39c9e6262a5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:44.718189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:44.748948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ff514-903a-4ea3-803f-28a75b30d4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:44.760339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ff514-903a-4ea3-803f-28a75b30d4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:44.760635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ff514-903a-4ea3-803f-28a75b30d4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:44.760920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ff514-903a-4ea3-803f-28a75b30d4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:44.761676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ff514-903a-4ea3-803f-28a75b30d4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:44.763699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ff514-903a-4ea3-803f-28a75b30d4c6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:44.849641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df39b2a-74e3-41df-a68a-50e3d28088ad admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:44.850882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 865/1732] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:44 2022] POST /identity/v3/users => generated 283 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:41:44.874952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df39b2a-74e3-41df-a68a-50e3d28088ad admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:44.893390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a9e20-6f03-4f9f-bd35-7838f9550d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:44.894568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a9e20-6f03-4f9f-bd35-7838f9550d89 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:44.894975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a9e20-6f03-4f9f-bd35-7838f9550d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:44.896010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a9e20-6f03-4f9f-bd35-7838f9550d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:44.896928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a9e20-6f03-4f9f-bd35-7838f9550d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:44.899575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a9e20-6f03-4f9f-bd35-7838f9550d89 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:44.922833 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 868/1733] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:44.928351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ff514-903a-4ea3-803f-28a75b30d4c6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:44.943620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ff514-903a-4ea3-803f-28a75b30d4c6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:44.965337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d136f61-39b3-4960-86a5-599bce67bc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:44.966039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d136f61-39b3-4960-86a5-599bce67bc1a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:44.966965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d136f61-39b3-4960-86a5-599bce67bc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:44.967254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d136f61-39b3-4960-86a5-599bce67bc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:44.968144 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d136f61-39b3-4960-86a5-599bce67bc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:44.969793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d136f61-39b3-4960-86a5-599bce67bc1a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:45.015145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 866/1734] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:41:45.020076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a9e20-6f03-4f9f-bd35-7838f9550d89 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:45.030337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a9e20-6f03-4f9f-bd35-7838f9550d89 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:45.049591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69345d-5ad0-473e-8b47-c5f423d169c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:45.050504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69345d-5ad0-473e-8b47-c5f423d169c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:45.050786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69345d-5ad0-473e-8b47-c5f423d169c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:45.051066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69345d-5ad0-473e-8b47-c5f423d169c7 admin admin] PATH_INFO: `/v3/projects/7e5dbbe5e04c453d901367fe73c5feb3/users/c27dfd091c3a4e4bbb8d2da2b717dca3/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:45.090581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69345d-5ad0-473e-8b47-c5f423d169c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5dbbe5e04c453d901367fe73c5feb3, user_id=c27dfd091c3a4e4bbb8d2da2b717dca3, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:45.093176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69345d-5ad0-473e-8b47-c5f423d169c7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:45.123493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 869/1735] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:45 2022] PUT /identity/v3/projects/7e5dbbe5e04c453d901367fe73c5feb3/users/c27dfd091c3a4e4bbb8d2da2b717dca3/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:45.133921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce20b73-b213-468a-8e61-df697fc87569 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:45.134259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce20b73-b213-468a-8e61-df697fc87569 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:45.134479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce20b73-b213-468a-8e61-df697fc87569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:45.237703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cce20b73-b213-468a-8e61-df697fc87569 None None] MFA Rules not processed for user `c27dfd091c3a4e4bbb8d2da2b717dca3`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:45.240700 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cce20b73-b213-468a-8e61-df697fc87569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:45.342572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69345d-5ad0-473e-8b47-c5f423d169c7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:45.345229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6b69345d-5ad0-473e-8b47-c5f423d169c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:45.479087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 867/1736] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:45.483620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce20b73-b213-468a-8e61-df697fc87569 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:45.566969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69345d-5ad0-473e-8b47-c5f423d169c7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:45.587941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:45.589219 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:45.590392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:45.591351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] PATH_INFO: `/v3/projects/d6044e66e27f4b788c77eb76095ad3b8` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:45.596869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6044e66e27f4b788c77eb76095ad3b8)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:45.600687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:45.689918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce20b73-b213-468a-8e61-df697fc87569 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:45.691862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:45.706303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3492a2f-53fa-4191-a577-a0ca657e7bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:45.706994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3492a2f-53fa-4191-a577-a0ca657e7bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:45.711989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3492a2f-53fa-4191-a577-a0ca657e7bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:45.712677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3492a2f-53fa-4191-a577-a0ca657e7bc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:45.713595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3492a2f-53fa-4191-a577-a0ca657e7bc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:45.718027 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3492a2f-53fa-4191-a577-a0ca657e7bc3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:45.783588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 868/1737] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:45 2022] POST /identity/v3/projects => generated 365 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:41:45.787729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3492a2f-53fa-4191-a577-a0ca657e7bc3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:45.791057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6044e66e27f4b788c77eb76095ad3b8', 'request_id': 'req-568282c9-1055-4e10-aa8d-8606291fa52d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:45.813233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6044e66e27f4b788c77eb76095ad3b8', 'request_id': 'req-568282c9-1055-4e10-aa8d-8606291fa52d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:45.859352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6044e66e27f4b788c77eb76095ad3b8', 'request_id': 'req-568282c9-1055-4e10-aa8d-8606291fa52d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:45.889611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] The token cache is being invalidate because project d6044e66e27f4b788c77eb76095ad3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:45.889983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:45.890838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-568282c9-1055-4e10-aa8d-8606291fa52d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:45.898101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 870/1738] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:45 2022] DELETE /identity/v3/projects/d6044e66e27f4b788c77eb76095ad3b8 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:45.903103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c4af8-5f60-4435-8cac-ee1073ee258b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:45.903539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c4af8-5f60-4435-8cac-ee1073ee258b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:45.903922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c4af8-5f60-4435-8cac-ee1073ee258b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:45.962414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-567c4af8-5f60-4435-8cac-ee1073ee258b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:45.964043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-567c4af8-5f60-4435-8cac-ee1073ee258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:46.015957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:46.058272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3492a2f-53fa-4191-a577-a0ca657e7bc3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:46.087984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a39fc-8beb-4df7-a428-d81bd785614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:46.088690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a39fc-8beb-4df7-a428-d81bd785614b admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:46.088981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a39fc-8beb-4df7-a428-d81bd785614b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:46.089271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a39fc-8beb-4df7-a428-d81bd785614b admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:46.092032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a39fc-8beb-4df7-a428-d81bd785614b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:46.094047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a39fc-8beb-4df7-a428-d81bd785614b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:46.165774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 869/1739] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:45 2022] POST /identity/v3/users => generated 289 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:41:46.174376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a39fc-8beb-4df7-a428-d81bd785614b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:46.181423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b07a39fc-8beb-4df7-a428-d81bd785614b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:46.229884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 871/1740] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:46.230096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a39fc-8beb-4df7-a428-d81bd785614b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:46.232200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958cae0-d3e1-426c-a366-8acd46cec045 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:46.232643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958cae0-d3e1-426c-a366-8acd46cec045 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:46.232938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958cae0-d3e1-426c-a366-8acd46cec045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:46.264725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c83cf-1eff-4929-b81c-4840f67e13bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:46.265421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c83cf-1eff-4929-b81c-4840f67e13bc admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:46.265710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c83cf-1eff-4929-b81c-4840f67e13bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:46.265991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c83cf-1eff-4929-b81c-4840f67e13bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:46.274963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c83cf-1eff-4929-b81c-4840f67e13bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:46.276806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c83cf-1eff-4929-b81c-4840f67e13bc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:46.292397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7958cae0-d3e1-426c-a366-8acd46cec045 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:46.294541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7958cae0-d3e1-426c-a366-8acd46cec045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:46.305339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 870/1741] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:46.310744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c83cf-1eff-4929-b81c-4840f67e13bc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:46.326179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 872/1742] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:46.327898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958cae0-d3e1-426c-a366-8acd46cec045 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:46.345968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c83cf-1eff-4929-b81c-4840f67e13bc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:46.354812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958cae0-d3e1-426c-a366-8acd46cec045 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:46.359640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e8a2f-c92a-484e-a893-0bf5e20bc8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:46.360557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e8a2f-c92a-484e-a893-0bf5e20bc8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:46.360842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e8a2f-c92a-484e-a893-0bf5e20bc8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:46.361173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e8a2f-c92a-484e-a893-0bf5e20bc8c7 admin admin] PATH_INFO: `/v3/projects/4443f8b6aa49499d93d9752ea5272187/users/e2bbab5c03714da98c683004c1a92a6e/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:46.368294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707407c-17d1-4e0f-94e7-017e258f770d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:46.368947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707407c-17d1-4e0f-94e7-017e258f770d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:46.369099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707407c-17d1-4e0f-94e7-017e258f770d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:46.369370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707407c-17d1-4e0f-94e7-017e258f770d admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:46.370235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707407c-17d1-4e0f-94e7-017e258f770d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:46.370419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707407c-17d1-4e0f-94e7-017e258f770d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:46.372079 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707407c-17d1-4e0f-94e7-017e258f770d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:46.414656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e8a2f-c92a-484e-a893-0bf5e20bc8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4443f8b6aa49499d93d9752ea5272187, user_id=e2bbab5c03714da98c683004c1a92a6e, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:46.422148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e8a2f-c92a-484e-a893-0bf5e20bc8c7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:46.429159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 873/1743] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:41:46.430205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707407c-17d1-4e0f-94e7-017e258f770d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:46.451497 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 871/1744] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:46 2022] PUT /identity/v3/projects/4443f8b6aa49499d93d9752ea5272187/users/e2bbab5c03714da98c683004c1a92a6e/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:46.456768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e8a2f-c92a-484e-a893-0bf5e20bc8c7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:46.458013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707407c-17d1-4e0f-94e7-017e258f770d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:46.676616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e8a2f-c92a-484e-a893-0bf5e20bc8c7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:46.699238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14590e-cb61-407c-9428-11b6334b3af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:46.700932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14590e-cb61-407c-9428-11b6334b3af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:46.709516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14590e-cb61-407c-9428-11b6334b3af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:46.711918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14590e-cb61-407c-9428-11b6334b3af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:46.713381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14590e-cb61-407c-9428-11b6334b3af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:46.717225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d831fa0-754e-4359-877c-bdf4aedf304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:46.718046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d831fa0-754e-4359-877c-bdf4aedf304e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:46.719925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d831fa0-754e-4359-877c-bdf4aedf304e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:46.720445 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d831fa0-754e-4359-877c-bdf4aedf304e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:46.720572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14590e-cb61-407c-9428-11b6334b3af8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:46.721522 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d831fa0-754e-4359-877c-bdf4aedf304e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:46.723774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d831fa0-754e-4359-877c-bdf4aedf304e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:46.802564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 872/1745] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:46.804560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d831fa0-754e-4359-877c-bdf4aedf304e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:46.830492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d831fa0-754e-4359-877c-bdf4aedf304e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:46.841222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d065d67-6081-43ca-8de0-5717f0ffd391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:46.844997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d065d67-6081-43ca-8de0-5717f0ffd391 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:46.846049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d065d67-6081-43ca-8de0-5717f0ffd391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:46.846464 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d065d67-6081-43ca-8de0-5717f0ffd391 admin admin] PATH_INFO: `/v3/projects/4443f8b6aa49499d93d9752ea5272187/users/e2bbab5c03714da98c683004c1a92a6e/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:46.849596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 874/1746] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:46 2022] POST /identity/v3/projects => generated 379 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:46.850869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d065d67-6081-43ca-8de0-5717f0ffd391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4443f8b6aa49499d93d9752ea5272187, user_id=e2bbab5c03714da98c683004c1a92a6e, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:46.855528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d065d67-6081-43ca-8de0-5717f0ffd391 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:46.858857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14590e-cb61-407c-9428-11b6334b3af8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:46.889267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:46.894353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 873/1747] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:46 2022] PUT /identity/v3/projects/4443f8b6aa49499d93d9752ea5272187/users/e2bbab5c03714da98c683004c1a92a6e/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:46.906514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309c95-1fa8-4de3-8fad-358775a586d9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:46.906815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309c95-1fa8-4de3-8fad-358775a586d9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:46.907089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309c95-1fa8-4de3-8fad-358775a586d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:47.040766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e4309c95-1fa8-4de3-8fad-358775a586d9 None None] MFA Rules not processed for user `e2bbab5c03714da98c683004c1a92a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:47.045877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e4309c95-1fa8-4de3-8fad-358775a586d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:47.099853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14590e-cb61-407c-9428-11b6334b3af8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:47.281653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 874/1748] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:47.290296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309c95-1fa8-4de3-8fad-358775a586d9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:47.291919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e4309c95-1fa8-4de3-8fad-358775a586d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:47.298434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779257b5-20aa-4bc3-8dde-f650bc59f9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:47.299174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779257b5-20aa-4bc3-8dde-f650bc59f9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:47.299478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779257b5-20aa-4bc3-8dde-f650bc59f9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:47.300575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779257b5-20aa-4bc3-8dde-f650bc59f9f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:47.300878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779257b5-20aa-4bc3-8dde-f650bc59f9f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:47.307946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779257b5-20aa-4bc3-8dde-f650bc59f9f8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:47.316351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:47.375082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 875/1749] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:46 2022] POST /identity/v3/users => generated 290 bytes in 516 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:41:47.376351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779257b5-20aa-4bc3-8dde-f650bc59f9f8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:47.394792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779257b5-20aa-4bc3-8dde-f650bc59f9f8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:47.411783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309c95-1fa8-4de3-8fad-358775a586d9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:47.419711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df88b1d-88ae-4b27-a7c3-d99408229c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:47.420754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88b1d-88ae-4b27-a7c3-d99408229c39 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:47.421265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88b1d-88ae-4b27-a7c3-d99408229c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:47.421752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88b1d-88ae-4b27-a7c3-d99408229c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:47.423687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9593ea-6e2d-4435-b50a-a4818f55278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:47.424394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9593ea-6e2d-4435-b50a-a4818f55278b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:47.424685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9593ea-6e2d-4435-b50a-a4818f55278b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:47.424967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9593ea-6e2d-4435-b50a-a4818f55278b admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:47.426133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9593ea-6e2d-4435-b50a-a4818f55278b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:47.428482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9593ea-6e2d-4435-b50a-a4818f55278b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:47.442381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df88b1d-88ae-4b27-a7c3-d99408229c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:47.451486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df88b1d-88ae-4b27-a7c3-d99408229c39 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:47.460894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 876/1750] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:47.466081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9593ea-6e2d-4435-b50a-a4818f55278b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:47.469854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 875/1751] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:41:47.520762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9593ea-6e2d-4435-b50a-a4818f55278b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:47.548389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9e964-a535-4998-b5d2-d97e68d46e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:47.549087 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9e964-a535-4998-b5d2-d97e68d46e04 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:47.549530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9e964-a535-4998-b5d2-d97e68d46e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:47.556493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9e964-a535-4998-b5d2-d97e68d46e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:47.557761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9e964-a535-4998-b5d2-d97e68d46e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:47.560179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9e964-a535-4998-b5d2-d97e68d46e04 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:47.632242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df88b1d-88ae-4b27-a7c3-d99408229c39 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:47.632927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 877/1752] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:47.634404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9e964-a535-4998-b5d2-d97e68d46e04 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:47.650623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df88b1d-88ae-4b27-a7c3-d99408229c39 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:47.669029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddcd00-95dc-4f4f-be16-1c1744afbeed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:47.669842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddcd00-95dc-4f4f-be16-1c1744afbeed service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:47.670243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddcd00-95dc-4f4f-be16-1c1744afbeed service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:47.670628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddcd00-95dc-4f4f-be16-1c1744afbeed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:47.682010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f2ddcd00-95dc-4f4f-be16-1c1744afbeed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:47.703946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9e964-a535-4998-b5d2-d97e68d46e04 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:47.717464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f6b0-0efb-41a6-a58e-e1aa89409256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:47.718392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f6b0-0efb-41a6-a58e-e1aa89409256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:47.718675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f6b0-0efb-41a6-a58e-e1aa89409256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:47.718944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f6b0-0efb-41a6-a58e-e1aa89409256 admin admin] PATH_INFO: `/v3/projects/e19722eed8d64c86b48c84fc08e41313/users/83a45d413fa94ec9a361cf73e93e68ba/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:47.820585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817f6b0-0efb-41a6-a58e-e1aa89409256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19722eed8d64c86b48c84fc08e41313, user_id=83a45d413fa94ec9a361cf73e93e68ba, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:47.828737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817f6b0-0efb-41a6-a58e-e1aa89409256 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:47.896251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 878/1753] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:47 2022] PUT /identity/v3/projects/e19722eed8d64c86b48c84fc08e41313/users/83a45d413fa94ec9a361cf73e93e68ba/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:47.897835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70187783-f9c9-4958-9c33-77d876ac1f31 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:47.898173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70187783-f9c9-4958-9c33-77d876ac1f31 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:47.898730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70187783-f9c9-4958-9c33-77d876ac1f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:47.936303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddcd00-95dc-4f4f-be16-1c1744afbeed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:47.948693 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-70187783-f9c9-4958-9c33-77d876ac1f31 None None] MFA Rules not processed for user `0b22e96941dd414da4fa22ed4bd36984`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:47.952455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-70187783-f9c9-4958-9c33-77d876ac1f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:47.955163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddcd00-95dc-4f4f-be16-1c1744afbeed service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:47.976536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 876/1754] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:47.978867 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2282182-49e4-44dc-bd30-ad999b5feda3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:47.979318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2282182-49e4-44dc-bd30-ad999b5feda3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:47.979668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2282182-49e4-44dc-bd30-ad999b5feda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:48.028444 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c2282182-49e4-44dc-bd30-ad999b5feda3 None None] MFA Rules not processed for user `83a45d413fa94ec9a361cf73e93e68ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:48.030037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c2282182-49e4-44dc-bd30-ad999b5feda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:48.135911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 879/1755] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:48.167070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 877/1756] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:48.171725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70187783-f9c9-4958-9c33-77d876ac1f31 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:48.188943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:48.410330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70187783-f9c9-4958-9c33-77d876ac1f31 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:48.424761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4aee0-33ac-498b-85c7-ee347f348f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:48.428530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4aee0-33ac-498b-85c7-ee347f348f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:48.428840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4aee0-33ac-498b-85c7-ee347f348f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:48.429129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4aee0-33ac-498b-85c7-ee347f348f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:48.429849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4aee0-33ac-498b-85c7-ee347f348f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:48.431841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4aee0-33ac-498b-85c7-ee347f348f89 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:48.512901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 880/1757] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:48 2022] POST /identity/v3/projects => generated 379 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:48.513875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2282182-49e4-44dc-bd30-ad999b5feda3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:48.536696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2282182-49e4-44dc-bd30-ad999b5feda3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:48.655107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:48.682174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2674944-5a2b-4bb9-a419-7709e8baecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:48.683108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2674944-5a2b-4bb9-a419-7709e8baecc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:48.685537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2674944-5a2b-4bb9-a419-7709e8baecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:48.685976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2674944-5a2b-4bb9-a419-7709e8baecc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:48.686886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2674944-5a2b-4bb9-a419-7709e8baecc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:48.690680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2674944-5a2b-4bb9-a419-7709e8baecc9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:48.762438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 878/1758] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:48 2022] POST /identity/v3/users => generated 290 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:41:48.766509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4aee0-33ac-498b-85c7-ee347f348f89 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:48.789734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4aee0-33ac-498b-85c7-ee347f348f89 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:48.800715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec483c-b2db-40ed-92cd-f52bc39c36c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:48.801688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec483c-b2db-40ed-92cd-f52bc39c36c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:48.808231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec483c-b2db-40ed-92cd-f52bc39c36c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:48.808927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec483c-b2db-40ed-92cd-f52bc39c36c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:48.810047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec483c-b2db-40ed-92cd-f52bc39c36c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:48.814105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec483c-b2db-40ed-92cd-f52bc39c36c3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:48.856153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 881/1759] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:48.860045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2674944-5a2b-4bb9-a419-7709e8baecc9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:48.873259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2674944-5a2b-4bb9-a419-7709e8baecc9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:48.902378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d4df5-b3c7-46aa-bacf-6a0150fc184a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:48.904077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d4df5-b3c7-46aa-bacf-6a0150fc184a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:48.904507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d4df5-b3c7-46aa-bacf-6a0150fc184a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:48.905563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d4df5-b3c7-46aa-bacf-6a0150fc184a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:48.908873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d4df5-b3c7-46aa-bacf-6a0150fc184a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:48.910779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d4df5-b3c7-46aa-bacf-6a0150fc184a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:48.952495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 879/1760] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:48.953619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec483c-b2db-40ed-92cd-f52bc39c36c3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:48.968437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec483c-b2db-40ed-92cd-f52bc39c36c3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:48.986463 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78407871-4c1a-48d8-9f07-264691df0f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:48.987389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407871-4c1a-48d8-9f07-264691df0f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:48.987701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407871-4c1a-48d8-9f07-264691df0f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:48.987980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407871-4c1a-48d8-9f07-264691df0f20 admin admin] PATH_INFO: `/v3/projects/98f0a7262799432694e6578838d558f6/users/c036b3f6937f4ae0ab2ea1b307f779e9/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:48.991007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df0899-58eb-4be5-872c-ee2ef4cc240e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:48.991577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df0899-58eb-4be5-872c-ee2ef4cc240e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:48.992049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df0899-58eb-4be5-872c-ee2ef4cc240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:49.046271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78407871-4c1a-48d8-9f07-264691df0f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f0a7262799432694e6578838d558f6, user_id=c036b3f6937f4ae0ab2ea1b307f779e9, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:49.054516 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-77df0899-58eb-4be5-872c-ee2ef4cc240e None None] MFA Rules not processed for user `3b4865ad7a6d43b1b13593a142c2dec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:49.054709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78407871-4c1a-48d8-9f07-264691df0f20 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:49.056146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-77df0899-58eb-4be5-872c-ee2ef4cc240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:49.107164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 882/1761] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:48 2022] PUT /identity/v3/projects/98f0a7262799432694e6578838d558f6/users/c036b3f6937f4ae0ab2ea1b307f779e9/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:49.126173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1f2b-2abd-4200-950f-cf10a444409c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:49.126929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1f2b-2abd-4200-950f-cf10a444409c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:49.127440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1f2b-2abd-4200-950f-cf10a444409c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:49.217700 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e1df1f2b-2abd-4200-950f-cf10a444409c None None] MFA Rules not processed for user `c036b3f6937f4ae0ab2ea1b307f779e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:49.221338 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e1df1f2b-2abd-4200-950f-cf10a444409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:49.383586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 880/1762] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:41:49.397674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceba0c7-30d0-4553-8a97-919833c500a1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:49.397974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceba0c7-30d0-4553-8a97-919833c500a1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:49.398260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceba0c7-30d0-4553-8a97-919833c500a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:49.446608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1ceba0c7-30d0-4553-8a97-919833c500a1 None None] MFA Rules not processed for user `3b4865ad7a6d43b1b13593a142c2dec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:49.450255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1ceba0c7-30d0-4553-8a97-919833c500a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:49.475592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 883/1763] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:49.480572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1f2b-2abd-4200-950f-cf10a444409c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:49.597325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:49.632139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1f2b-2abd-4200-950f-cf10a444409c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:49.647039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41cef2-442c-476f-aedf-dca20cf8fdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:49.659980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41cef2-442c-476f-aedf-dca20cf8fdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:49.662529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41cef2-442c-476f-aedf-dca20cf8fdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:49.663013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41cef2-442c-476f-aedf-dca20cf8fdd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:49.664023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41cef2-442c-476f-aedf-dca20cf8fdd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:49.669514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41cef2-442c-476f-aedf-dca20cf8fdd0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:49.733953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 884/1764] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:49 2022] POST /identity/v3/projects => generated 379 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:41:49.735606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 881/1765] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:49.741187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6db9a-8ec9-43ce-bdbd-c06ebc8af499 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:49.746538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6db9a-8ec9-43ce-bdbd-c06ebc8af499 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:49.746965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6db9a-8ec9-43ce-bdbd-c06ebc8af499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:49.747355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceba0c7-30d0-4553-8a97-919833c500a1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:49.794084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceba0c7-30d0-4553-8a97-919833c500a1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:49.868099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6cf6db9a-8ec9-43ce-bdbd-c06ebc8af499 None None] MFA Rules not processed for user `c54ec6918ad44a31ba7137abb56b89b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:49.871166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6cf6db9a-8ec9-43ce-bdbd-c06ebc8af499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:50.049228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:50.082694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4bb97-30d9-4e6a-b055-b2b4c2242539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:50.087332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4bb97-30d9-4e6a-b055-b2b4c2242539 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:50.087955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4bb97-30d9-4e6a-b055-b2b4c2242539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:50.088523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4bb97-30d9-4e6a-b055-b2b4c2242539 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:50.089476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4bb97-30d9-4e6a-b055-b2b4c2242539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:50.098281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4bb97-30d9-4e6a-b055-b2b4c2242539 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:50.149709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 885/1766] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:50.150561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6db9a-8ec9-43ce-bdbd-c06ebc8af499 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:50.153409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6cf6db9a-8ec9-43ce-bdbd-c06ebc8af499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:50.196057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 882/1767] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:49 2022] POST /identity/v3/users => generated 296 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:50.197380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4bb97-30d9-4e6a-b055-b2b4c2242539 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:50.211015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-31f4bb97-30d9-4e6a-b055-b2b4c2242539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:50.251482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4bb97-30d9-4e6a-b055-b2b4c2242539 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:50.270962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:50.271964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:50.272247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:50.272560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] PATH_INFO: `/v3/users/7705af78464944038f8017435a7401ae` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:50.290284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7705af78464944038f8017435a7401ae)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:50.292877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:50.415419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7705af78464944038f8017435a7401ae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:50.418734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7705af78464944038f8017435a7401ae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:50.462478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7705af78464944038f8017435a7401ae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:50.464724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6db9a-8ec9-43ce-bdbd-c06ebc8af499 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:50.478568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:50.479232 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7705af78464944038f8017435a7401ae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:50.479324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:50.479497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:50.479789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:50.486837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:50.501528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-64b5f87b-b316-410d-8664-178dc9d142f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7705af78464944038f8017435a7401ae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:50.531428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 883/1768] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:50 2022] DELETE /identity/v3/users/7705af78464944038f8017435a7401ae => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:50.541185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5c3b4-abd1-4b38-9a9b-f91e448ff823 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:50.541498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5c3b4-abd1-4b38-9a9b-f91e448ff823 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:50.541972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5c3b4-abd1-4b38-9a9b-f91e448ff823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:50.674218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bbe5c3b4-abd1-4b38-9a9b-f91e448ff823 None None] MFA Rules not processed for user `c57d44c19d15496e96c6e37510ce67fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:50.676283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bbe5c3b4-abd1-4b38-9a9b-f91e448ff823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:50.723167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:50.728925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:50.742789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:50.753923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 886/1769] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:50.755983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:50.764075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:50.788082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33356f7-3364-4d77-8841-3a001a0ce347 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:50.837377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 884/1770] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Jul 13 20:41:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:50.841803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5c3b4-abd1-4b38-9a9b-f91e448ff823 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:50.843370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bbe5c3b4-abd1-4b38-9a9b-f91e448ff823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:50.991863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:50.992746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:50.993174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:50.993590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.000655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:51.008628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5c3b4-abd1-4b38-9a9b-f91e448ff823 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:51.013965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5af365-94cd-417a-9ce9-1e561981d371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:51.015048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5af365-94cd-417a-9ce9-1e561981d371 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.015441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5af365-94cd-417a-9ce9-1e561981d371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.015849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5af365-94cd-417a-9ce9-1e561981d371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.017470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5af365-94cd-417a-9ce9-1e561981d371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:51.019554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5af365-94cd-417a-9ce9-1e561981d371 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:51.037084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 885/1771] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:51.039249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c588a-ab4b-4fbb-9daf-2903579f1705 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.039822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c588a-ab4b-4fbb-9daf-2903579f1705 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.041566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c588a-ab4b-4fbb-9daf-2903579f1705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.079898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:51.136073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1b6c588a-ab4b-4fbb-9daf-2903579f1705 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:51.137444 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1b6c588a-ab4b-4fbb-9daf-2903579f1705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:51.186398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 886/1772] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:51.192606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c588a-ab4b-4fbb-9daf-2903579f1705 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:51.230333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c588a-ab4b-4fbb-9daf-2903579f1705 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:51.256012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:51.257639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:51.264071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c4f10-cfae-4fac-b3a8-c0668d444153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:51.264482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c4f10-cfae-4fac-b3a8-c0668d444153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.264770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c4f10-cfae-4fac-b3a8-c0668d444153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.265047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c4f10-cfae-4fac-b3a8-c0668d444153 admin admin] PATH_INFO: `/v3/projects/a18edc9e4ea84bd08d9202bc292d29b1/users/f7224be8b49a414987898221eacb2283/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.302977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 887/1773] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:51.304132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:51.314480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:51.328926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c4f10-cfae-4fac-b3a8-c0668d444153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18edc9e4ea84bd08d9202bc292d29b1, user_id=f7224be8b49a414987898221eacb2283, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:51.331694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c4f10-cfae-4fac-b3a8-c0668d444153 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:51.348705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277e3bb-ec7f-465b-85fa-a502a47c6152 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:51.357086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5bf3f-47ca-48a8-ab64-a208f3fb9d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:51.357764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5bf3f-47ca-48a8-ab64-a208f3fb9d5e service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.358055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5bf3f-47ca-48a8-ab64-a208f3fb9d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.358331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5bf3f-47ca-48a8-ab64-a208f3fb9d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.366363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 887/1774] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:51 2022] PUT /identity/v3/projects/a18edc9e4ea84bd08d9202bc292d29b1/users/f7224be8b49a414987898221eacb2283/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:51.370451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c4f10-cfae-4fac-b3a8-c0668d444153 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:51.399491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5bf3f-47ca-48a8-ab64-a208f3fb9d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:51.402344 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5bf3f-47ca-48a8-ab64-a208f3fb9d5e service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:51.438491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 888/1775] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:51.446525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a32ff-3fc6-4324-8825-f2c0f67941fb None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.446836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a32ff-3fc6-4324-8825-f2c0f67941fb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.447125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a32ff-3fc6-4324-8825-f2c0f67941fb None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.464906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 889/1776] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 13 20:41:51 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:41:51.470741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a32ff-3fc6-4324-8825-f2c0f67941fb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:51.480595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1e6a32ff-3fc6-4324-8825-f2c0f67941fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:51.549108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c4f10-cfae-4fac-b3a8-c0668d444153 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:51.574198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced7e7-e35c-4302-8d83-aab048b46944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:51.576035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced7e7-e35c-4302-8d83-aab048b46944 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.576492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced7e7-e35c-4302-8d83-aab048b46944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.577329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced7e7-e35c-4302-8d83-aab048b46944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.578652 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced7e7-e35c-4302-8d83-aab048b46944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:51.583032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced7e7-e35c-4302-8d83-aab048b46944 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:51.616794 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 888/1777] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:51.626203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7af2-e0fa-4790-9f9b-9ac13c1ee0b5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.626577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7af2-e0fa-4790-9f9b-9ac13c1ee0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.632575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7af2-e0fa-4790-9f9b-9ac13c1ee0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.688194 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ceea7af2-e0fa-4790-9f9b-9ac13c1ee0b5 None None] MFA Rules not processed for user `c57d44c19d15496e96c6e37510ce67fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:51.688532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a32ff-3fc6-4324-8825-f2c0f67941fb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:51.689418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ceea7af2-e0fa-4790-9f9b-9ac13c1ee0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:51.707674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2b58d-2500-44ec-9da2-903a5dda6ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:51.710630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2b58d-2500-44ec-9da2-903a5dda6ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.711880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2b58d-2500-44ec-9da2-903a5dda6ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.712179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2b58d-2500-44ec-9da2-903a5dda6ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.723907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 889/1778] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:41:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:51.724753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea7af2-e0fa-4790-9f9b-9ac13c1ee0b5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:51.743620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2b58d-2500-44ec-9da2-903a5dda6ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:51.752282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2b58d-2500-44ec-9da2-903a5dda6ed5 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:51.763904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea7af2-e0fa-4790-9f9b-9ac13c1ee0b5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:51.776059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 890/1779] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:51.780018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564bcaf-cfca-40af-a437-f20850a62cbd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.781391 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564bcaf-cfca-40af-a437-f20850a62cbd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.781684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564bcaf-cfca-40af-a437-f20850a62cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.787620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999d29c-f9be-47cb-a1ca-9917c6211e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:51.788549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999d29c-f9be-47cb-a1ca-9917c6211e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:51.788992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999d29c-f9be-47cb-a1ca-9917c6211e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:51.789084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999d29c-f9be-47cb-a1ca-9917c6211e51 admin admin] PATH_INFO: `/v3/projects/a18edc9e4ea84bd08d9202bc292d29b1/users/f7224be8b49a414987898221eacb2283/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:51.792483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999d29c-f9be-47cb-a1ca-9917c6211e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18edc9e4ea84bd08d9202bc292d29b1, user_id=f7224be8b49a414987898221eacb2283, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:51.794856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999d29c-f9be-47cb-a1ca-9917c6211e51 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:51.837900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 890/1780] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:51 2022] PUT /identity/v3/projects/a18edc9e4ea84bd08d9202bc292d29b1/users/f7224be8b49a414987898221eacb2283/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:51.838560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999d29c-f9be-47cb-a1ca-9917c6211e51 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:51.870063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1564bcaf-cfca-40af-a437-f20850a62cbd None None] MFA Rules not processed for user `798de1406eac40b6bc5cf2f5bea4846f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:51.871302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1564bcaf-cfca-40af-a437-f20850a62cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:51.907988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:52.026660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 891/1781] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:52.028373 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f654a8-ebd0-44b0-b1a2-20ae14bf6cd0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:52.030178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f654a8-ebd0-44b0-b1a2-20ae14bf6cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:52.030469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f654a8-ebd0-44b0-b1a2-20ae14bf6cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:52.067594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999d29c-f9be-47cb-a1ca-9917c6211e51 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:52.069156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-90f654a8-ebd0-44b0-b1a2-20ae14bf6cd0 None None] MFA Rules not processed for user `f7224be8b49a414987898221eacb2283`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:52.070706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-90f654a8-ebd0-44b0-b1a2-20ae14bf6cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:52.084297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41858222-225e-4f2a-8be4-240d23ba5242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:52.085830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41858222-225e-4f2a-8be4-240d23ba5242 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:52.085830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41858222-225e-4f2a-8be4-240d23ba5242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:52.085830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41858222-225e-4f2a-8be4-240d23ba5242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:52.105089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41858222-225e-4f2a-8be4-240d23ba5242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:52.106755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41858222-225e-4f2a-8be4-240d23ba5242 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:52.145581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 891/1782] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:52.146915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41853e37-37fe-4fb8-b757-355ad7c119ff None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:52.147208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41853e37-37fe-4fb8-b757-355ad7c119ff None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:52.156394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41853e37-37fe-4fb8-b757-355ad7c119ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:52.210172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-41853e37-37fe-4fb8-b757-355ad7c119ff None None] MFA Rules not processed for user `798de1406eac40b6bc5cf2f5bea4846f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:52.211427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-41853e37-37fe-4fb8-b757-355ad7c119ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:52.232860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 892/1783] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:52.330865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 892/1784] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:52.342252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41853e37-37fe-4fb8-b757-355ad7c119ff None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:52.344485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-41853e37-37fe-4fb8-b757-355ad7c119ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:52.505183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:52.534084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41853e37-37fe-4fb8-b757-355ad7c119ff None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:52.567121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89346006-cbf3-48f4-a5a1-9d1309779553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:52.568189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89346006-cbf3-48f4-a5a1-9d1309779553 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:52.568462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89346006-cbf3-48f4-a5a1-9d1309779553 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:52.568738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89346006-cbf3-48f4-a5a1-9d1309779553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:52.569352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f654a8-ebd0-44b0-b1a2-20ae14bf6cd0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:52.572581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-90f654a8-ebd0-44b0-b1a2-20ae14bf6cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:52.612958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89346006-cbf3-48f4-a5a1-9d1309779553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:52.614470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89346006-cbf3-48f4-a5a1-9d1309779553 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:52.630586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 893/1785] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:52.773926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89346006-cbf3-48f4-a5a1-9d1309779553 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:52.775568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-89346006-cbf3-48f4-a5a1-9d1309779553 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:52.801467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89346006-cbf3-48f4-a5a1-9d1309779553 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:52.836113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798b727-9258-42f2-86dd-e58fd010ee70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:52.836807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798b727-9258-42f2-86dd-e58fd010ee70 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:52.839985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798b727-9258-42f2-86dd-e58fd010ee70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:52.840403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798b727-9258-42f2-86dd-e58fd010ee70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:52.845676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2798b727-9258-42f2-86dd-e58fd010ee70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:52.860278 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f654a8-ebd0-44b0-b1a2-20ae14bf6cd0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:52.897959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:52.901128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:52.901128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:52.901128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] PATH_INFO: `/v3/users/c27dfd091c3a4e4bbb8d2da2b717dca3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:52.917696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c27dfd091c3a4e4bbb8d2da2b717dca3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:52.927491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:53.011521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798b727-9258-42f2-86dd-e58fd010ee70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:53.013219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798b727-9258-42f2-86dd-e58fd010ee70 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:53.034666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 894/1786] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:53.056662 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798b727-9258-42f2-86dd-e58fd010ee70 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:53.069309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:53.087405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c27dfd091c3a4e4bbb8d2da2b717dca3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:53.089192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c27dfd091c3a4e4bbb8d2da2b717dca3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:53.132012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c27dfd091c3a4e4bbb8d2da2b717dca3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:53.184934 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c27dfd091c3a4e4bbb8d2da2b717dca3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:53.186127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-82e84634-9e02-4106-9735-90283c1ead24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c27dfd091c3a4e4bbb8d2da2b717dca3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:53.226846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 893/1787] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:52 2022] DELETE /identity/v3/users/c27dfd091c3a4e4bbb8d2da2b717dca3 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:53.231644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bc122-d7ab-43c0-8f1e-53f65a0b266e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:53.232284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bc122-d7ab-43c0-8f1e-53f65a0b266e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:53.232826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bc122-d7ab-43c0-8f1e-53f65a0b266e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:53.261475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798b727-9258-42f2-86dd-e58fd010ee70 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:53.281422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-195bc122-d7ab-43c0-8f1e-53f65a0b266e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:53.285053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-195bc122-d7ab-43c0-8f1e-53f65a0b266e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:53.496269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:53.497062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:53.497462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:53.497841 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:53.507601 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:53.558921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 894/1788] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:53.609278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:53.615768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bc122-d7ab-43c0-8f1e-53f65a0b266e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:53.619001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-195bc122-d7ab-43c0-8f1e-53f65a0b266e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:53.749613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:53.751762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:53.808081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 895/1789] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 754 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:41:53.867197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bc122-d7ab-43c0-8f1e-53f65a0b266e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:53.879602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2ad3a-f6e7-43e9-b167-66abeee7eb2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:53.881568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2ad3a-f6e7-43e9-b167-66abeee7eb2f service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:53.881883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2ad3a-f6e7-43e9-b167-66abeee7eb2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:53.883953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2ad3a-f6e7-43e9-b167-66abeee7eb2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:53.928382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2ad3a-f6e7-43e9-b167-66abeee7eb2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:53.932400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2ad3a-f6e7-43e9-b167-66abeee7eb2f service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:54.014251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 895/1790] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:54.089427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:54.091120 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:54.125423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120398f-38ac-41f3-aec0-a5558aded6d4 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:54.142436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:54.144173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:54.144666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:54.145152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] PATH_INFO: `/v3/projects/557486c3564a40bc8476717f908d3b48` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:54.167900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=557486c3564a40bc8476717f908d3b48)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:54.170131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:54.361093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557486c3564a40bc8476717f908d3b48', 'request_id': 'req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.382047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557486c3564a40bc8476717f908d3b48', 'request_id': 'req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.400572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557486c3564a40bc8476717f908d3b48', 'request_id': 'req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.437584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] The token cache is being invalidate because project 557486c3564a40bc8476717f908d3b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:54.438764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.441321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6ce16bb0-76e4-4a0b-b7a5-8b0d44f5dd13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.449875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 896/1791] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:54 2022] DELETE /identity/v3/projects/557486c3564a40bc8476717f908d3b48 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:54.458748 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2ad3a-f6e7-43e9-b167-66abeee7eb2f service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:54.472846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2ad3a-f6e7-43e9-b167-66abeee7eb2f service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:54.635161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:54.637984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:54.639824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:54.640376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] PATH_INFO: `/v3/users/2c13d340b2fe416db7948576b6441b99` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:54.643750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c13d340b2fe416db7948576b6441b99)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:54.648164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:54.656497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:54.756071 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c13d340b2fe416db7948576b6441b99'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.757576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c13d340b2fe416db7948576b6441b99'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.787235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c13d340b2fe416db7948576b6441b99'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.799181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c13d340b2fe416db7948576b6441b99'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.801741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e666821d-58bd-40bd-ba6b-7c05db1dd7ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c13d340b2fe416db7948576b6441b99'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:54.834834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 896/1792] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:54 2022] DELETE /identity/v3/users/2c13d340b2fe416db7948576b6441b99 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:56.089011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382d220-e355-4ef4-9be3-914376dcdcf1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:56.089630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382d220-e355-4ef4-9be3-914376dcdcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:56.090807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382d220-e355-4ef4-9be3-914376dcdcf1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:56.094364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 897/1793] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:41:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:41:56.138119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb8117-75e6-4062-b8ed-c36323af56b9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:56.138119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb8117-75e6-4062-b8ed-c36323af56b9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:56.138119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb8117-75e6-4062-b8ed-c36323af56b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:56.204804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c3cb8117-75e6-4062-b8ed-c36323af56b9 None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:56.206507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c3cb8117-75e6-4062-b8ed-c36323af56b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:56.299958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382d220-e355-4ef4-9be3-914376dcdcf1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:56.307451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4382d220-e355-4ef4-9be3-914376dcdcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:56.509673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 897/1794] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:41:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:56.579538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382d220-e355-4ef4-9be3-914376dcdcf1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:56.592214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:56.593377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:56.593852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:56.595826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] PATH_INFO: `/v3/projects/7e5dbbe5e04c453d901367fe73c5feb3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:56.600301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5dbbe5e04c453d901367fe73c5feb3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:56.619207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:56.656778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb8117-75e6-4062-b8ed-c36323af56b9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:56.659220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c3cb8117-75e6-4062-b8ed-c36323af56b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:56.750776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb8117-75e6-4062-b8ed-c36323af56b9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:56.781327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9b3ee-f79c-421f-aedc-74ec5b5083a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:56.782036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9b3ee-f79c-421f-aedc-74ec5b5083a1 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:56.782450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9b3ee-f79c-421f-aedc-74ec5b5083a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:56.782792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9b3ee-f79c-421f-aedc-74ec5b5083a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:56.841317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9b3ee-f79c-421f-aedc-74ec5b5083a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:56.843702 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9b3ee-f79c-421f-aedc-74ec5b5083a1 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:56.872369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:56.876323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5dbbe5e04c453d901367fe73c5feb3', 'request_id': 'req-6cc85228-8150-47ef-bdb8-32ceefb7fb85'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:56.893630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 898/1795] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:56.900737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5dbbe5e04c453d901367fe73c5feb3', 'request_id': 'req-6cc85228-8150-47ef-bdb8-32ceefb7fb85'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:56.910803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5dbbe5e04c453d901367fe73c5feb3', 'request_id': 'req-6cc85228-8150-47ef-bdb8-32ceefb7fb85'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:56.929068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] The token cache is being invalidate because project 7e5dbbe5e04c453d901367fe73c5feb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:56.929470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:56.931000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:56.936877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 898/1796] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:56 2022] DELETE /identity/v3/projects/7e5dbbe5e04c453d901367fe73c5feb3 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:56.938069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9b3ee-f79c-421f-aedc-74ec5b5083a1 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:56.951665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9b3ee-f79c-421f-aedc-74ec5b5083a1 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:57.233953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:57.235797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:57.262233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:57.263253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:57.263612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:57.263903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] PATH_INFO: `/v3/users/e2bbab5c03714da98c683004c1a92a6e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:57.266443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc85228-8150-47ef-bdb8-32ceefb7fb85 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:57.270288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2bbab5c03714da98c683004c1a92a6e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:57.292446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:57.293756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:57.294338 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:57.296458 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:57.298392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:57.299887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] PATH_INFO: `/v3/users/83a45d413fa94ec9a361cf73e93e68ba` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:57.308751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a45d413fa94ec9a361cf73e93e68ba)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:57.319060 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:57.467289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2bbab5c03714da98c683004c1a92a6e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.468354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2bbab5c03714da98c683004c1a92a6e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.503918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a45d413fa94ec9a361cf73e93e68ba'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.506663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a45d413fa94ec9a361cf73e93e68ba'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.526995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2bbab5c03714da98c683004c1a92a6e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.550030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a45d413fa94ec9a361cf73e93e68ba'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.562585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2bbab5c03714da98c683004c1a92a6e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.569329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92b8a936-cc4f-46a9-9ba7-e06a7e81b816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2bbab5c03714da98c683004c1a92a6e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.573832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a45d413fa94ec9a361cf73e93e68ba'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.596289 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a45d413fa94ec9a361cf73e93e68ba'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:57.614749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 899/1797] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:56 2022] DELETE /identity/v3/users/e2bbab5c03714da98c683004c1a92a6e => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:57.622223 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 899/1798] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:41:57 2022] DELETE /identity/v3/users/83a45d413fa94ec9a361cf73e93e68ba => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:41:57.623095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6173c-4b7e-4063-9d20-fa31eee06398 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:57.623590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6173c-4b7e-4063-9d20-fa31eee06398 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:57.624012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6173c-4b7e-4063-9d20-fa31eee06398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:57.689924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-40f6173c-4b7e-4063-9d20-fa31eee06398 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:57.697413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-40f6173c-4b7e-4063-9d20-fa31eee06398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:57.839179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 900/1799] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:57.916211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:57.917612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:58.049619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c9cd2-7cc2-46a3-82e6-0ebb32252e97 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:58.054781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193371a-578f-42ce-862e-0fd3084e22d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:58.055483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193371a-578f-42ce-862e-0fd3084e22d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:58.055782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193371a-578f-42ce-862e-0fd3084e22d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:58.056062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193371a-578f-42ce-862e-0fd3084e22d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:58.072527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193371a-578f-42ce-862e-0fd3084e22d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:58.076347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193371a-578f-42ce-862e-0fd3084e22d3 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:58.093081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 900/1800] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:41:58.326915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6173c-4b7e-4063-9d20-fa31eee06398 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:58.329036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-40f6173c-4b7e-4063-9d20-fa31eee06398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:58.349339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6173c-4b7e-4063-9d20-fa31eee06398 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:58.360823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:58.361745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:58.362013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:58.362292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] PATH_INFO: `/v3/projects/269b18b56dc645c8b606ab6e78f3e5ce` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:58.363669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=269b18b56dc645c8b606ab6e78f3e5ce)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:58.365539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:58.479329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '269b18b56dc645c8b606ab6e78f3e5ce', 'request_id': 'req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:58.493893 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '269b18b56dc645c8b606ab6e78f3e5ce', 'request_id': 'req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:58.505533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '269b18b56dc645c8b606ab6e78f3e5ce', 'request_id': 'req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:58.530202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] The token cache is being invalidate because project 269b18b56dc645c8b606ab6e78f3e5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:41:58.530889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:58.533082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c647aa1-71f5-47ec-bdfb-79e2c61b87fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:41:58.543854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f3013-ff3e-48ae-8692-f17005890344 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:58.543854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f3013-ff3e-48ae-8692-f17005890344 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:58.543854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f3013-ff3e-48ae-8692-f17005890344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:58.544713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 901/1801] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:58 2022] DELETE /identity/v3/projects/269b18b56dc645c8b606ab6e78f3e5ce => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:58.576435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a24f3013-ff3e-48ae-8692-f17005890344 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:58.581276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a24f3013-ff3e-48ae-8692-f17005890344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:58.691248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:58.735074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53af0d-9065-4129-a8c2-53fa40ea108f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:58.735755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53af0d-9065-4129-a8c2-53fa40ea108f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:58.736150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53af0d-9065-4129-a8c2-53fa40ea108f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:58.738872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 901/1802] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:58.743241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:58.768228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dd53af0d-9065-4129-a8c2-53fa40ea108f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:58.770986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dd53af0d-9065-4129-a8c2-53fa40ea108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:58.796206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f3013-ff3e-48ae-8692-f17005890344 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:58.816732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 902/1803] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:58.823036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f3013-ff3e-48ae-8692-f17005890344 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:58.844284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4211c5a-28be-4751-8431-52d077f4d107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:58.845078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4211c5a-28be-4751-8431-52d077f4d107 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:58.845510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4211c5a-28be-4751-8431-52d077f4d107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:58.845778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4211c5a-28be-4751-8431-52d077f4d107 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:58.846598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4211c5a-28be-4751-8431-52d077f4d107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:41:58.846982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4211c5a-28be-4751-8431-52d077f4d107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:58.852190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4211c5a-28be-4751-8431-52d077f4d107 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:58.885062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53af0d-9065-4129-a8c2-53fa40ea108f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:58.890982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 902/1804] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:41:58.899908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53af0d-9065-4129-a8c2-53fa40ea108f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:58.911170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82f0aa-2029-4df5-b123-98c6e839d21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:58.911891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82f0aa-2029-4df5-b123-98c6e839d21b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:58.912180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82f0aa-2029-4df5-b123-98c6e839d21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:58.912761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82f0aa-2029-4df5-b123-98c6e839d21b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:58.913404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82f0aa-2029-4df5-b123-98c6e839d21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:58.915349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82f0aa-2029-4df5-b123-98c6e839d21b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:58.973655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 903/1805] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:41:58 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:41:58.978226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4211c5a-28be-4751-8431-52d077f4d107 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:58.993126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4211c5a-28be-4751-8431-52d077f4d107 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:59.106595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6290392-976e-4d08-b09f-916401201796 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:59.106970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6290392-976e-4d08-b09f-916401201796 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:59.107167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6290392-976e-4d08-b09f-916401201796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:59.136648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e6290392-976e-4d08-b09f-916401201796 None None] MFA Rules not processed for user `c54ec6918ad44a31ba7137abb56b89b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:59.139655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e6290392-976e-4d08-b09f-916401201796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:59.150808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373c95f-abf0-4d17-b087-aeb2ce08bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:59.151681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373c95f-abf0-4d17-b087-aeb2ce08bde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:59.152076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373c95f-abf0-4d17-b087-aeb2ce08bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:59.153155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373c95f-abf0-4d17-b087-aeb2ce08bde1 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:59.154734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373c95f-abf0-4d17-b087-aeb2ce08bde1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:59.160285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373c95f-abf0-4d17-b087-aeb2ce08bde1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:59.208644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 903/1806] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:41:58 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:41:59.212184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861ddd4-4102-49db-a1f7-161514e1eeb5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:59.213267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861ddd4-4102-49db-a1f7-161514e1eeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:59.213988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861ddd4-4102-49db-a1f7-161514e1eeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:59.263314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a861ddd4-4102-49db-a1f7-161514e1eeb5 None None] MFA Rules not processed for user `798de1406eac40b6bc5cf2f5bea4846f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:59.268758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a861ddd4-4102-49db-a1f7-161514e1eeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:59.316106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 904/1807] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:59.321160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6290392-976e-4d08-b09f-916401201796 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:59.359271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6290392-976e-4d08-b09f-916401201796 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:59.381693 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a07ff-c0e0-47f3-95aa-71485053c040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:59.384131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a07ff-c0e0-47f3-95aa-71485053c040 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:59.384461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a07ff-c0e0-47f3-95aa-71485053c040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:59.384771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a07ff-c0e0-47f3-95aa-71485053c040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:59.385532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a07ff-c0e0-47f3-95aa-71485053c040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:59.387377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a07ff-c0e0-47f3-95aa-71485053c040 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:59.440746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 905/1808] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:59.441751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a07ff-c0e0-47f3-95aa-71485053c040 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:59.447160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 904/1809] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:41:59.476328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a07ff-c0e0-47f3-95aa-71485053c040 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:59.502044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe717428-5574-42b4-bd0e-1ab73eb69d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:59.503408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe717428-5574-42b4-bd0e-1ab73eb69d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:59.503790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe717428-5574-42b4-bd0e-1ab73eb69d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:59.504113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe717428-5574-42b4-bd0e-1ab73eb69d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:59.505106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe717428-5574-42b4-bd0e-1ab73eb69d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:59.507146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe717428-5574-42b4-bd0e-1ab73eb69d67 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:59.526496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 906/1810] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:41:59.530204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861ddd4-4102-49db-a1f7-161514e1eeb5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:59.554490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861ddd4-4102-49db-a1f7-161514e1eeb5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:41:59.567217 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f955e1-d75e-4b5f-89a7-dae7e7141ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:41:59.568224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f955e1-d75e-4b5f-89a7-dae7e7141ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:59.568512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f955e1-d75e-4b5f-89a7-dae7e7141ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:59.568784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f955e1-d75e-4b5f-89a7-dae7e7141ac3 admin admin] PATH_INFO: `/v3/projects/ba92d95056ca4122b7462696ce2b43bb/users/2e7e652a39bd411b8dcdcaf87b7e635d/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:59.625268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f955e1-d75e-4b5f-89a7-dae7e7141ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba92d95056ca4122b7462696ce2b43bb, user_id=2e7e652a39bd411b8dcdcaf87b7e635d, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:41:59.628075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f955e1-d75e-4b5f-89a7-dae7e7141ac3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:41:59.672090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 905/1811] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:41:59 2022] PUT /identity/v3/projects/ba92d95056ca4122b7462696ce2b43bb/users/2e7e652a39bd411b8dcdcaf87b7e635d/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:41:59.682540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cec3fb-a12b-4712-9f0f-dab8f8ca9308 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:41:59.682888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cec3fb-a12b-4712-9f0f-dab8f8ca9308 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:41:59.683159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cec3fb-a12b-4712-9f0f-dab8f8ca9308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:41:59.730458 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f955e1-d75e-4b5f-89a7-dae7e7141ac3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:41:59.732662 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a5f955e1-d75e-4b5f-89a7-dae7e7141ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:59.773056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a1cec3fb-a12b-4712-9f0f-dab8f8ca9308 None None] MFA Rules not processed for user `2e7e652a39bd411b8dcdcaf87b7e635d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:41:59.774515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a1cec3fb-a12b-4712-9f0f-dab8f8ca9308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:41:59.874699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:41:59.991127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f955e1-d75e-4b5f-89a7-dae7e7141ac3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.004001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 907/1812] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:41:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:00.006479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cec3fb-a12b-4712-9f0f-dab8f8ca9308 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:00.010580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:00.011518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.011816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.012100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] PATH_INFO: `/v3/projects/4443f8b6aa49499d93d9752ea5272187` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.015099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=4443f8b6aa49499d93d9752ea5272187)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:00.016991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:00.043196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cec3fb-a12b-4712-9f0f-dab8f8ca9308 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.070823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c82f9-bfea-423a-a403-d9cb8ae2ddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:00.071571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c82f9-bfea-423a-a403-d9cb8ae2ddb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.071867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c82f9-bfea-423a-a403-d9cb8ae2ddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.075011 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c82f9-bfea-423a-a403-d9cb8ae2ddb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.075821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c82f9-bfea-423a-a403-d9cb8ae2ddb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:00.081524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c82f9-bfea-423a-a403-d9cb8ae2ddb7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:00.219187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 908/1813] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:00 2022] POST /identity/v3/projects => generated 381 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:00.225327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c82f9-bfea-423a-a403-d9cb8ae2ddb7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:00.248920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4443f8b6aa49499d93d9752ea5272187', 'request_id': 'req-68f07e3e-4766-401d-8004-3f048ff07bff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:00.282607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4443f8b6aa49499d93d9752ea5272187', 'request_id': 'req-68f07e3e-4766-401d-8004-3f048ff07bff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:00.304340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4443f8b6aa49499d93d9752ea5272187', 'request_id': 'req-68f07e3e-4766-401d-8004-3f048ff07bff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:00.331190 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:00.347742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] The token cache is being invalidate because project 4443f8b6aa49499d93d9752ea5272187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:00.348044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:00.355417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-68f07e3e-4766-401d-8004-3f048ff07bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:00.363695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 906/1814] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:41:59 2022] DELETE /identity/v3/projects/4443f8b6aa49499d93d9752ea5272187 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:00.373342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef7208-b723-4525-a629-b54e82d2b1e7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.373342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef7208-b723-4525-a629-b54e82d2b1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.373342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef7208-b723-4525-a629-b54e82d2b1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.391408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c82f9-bfea-423a-a403-d9cb8ae2ddb7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.398770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879038f-8b2b-49ed-8937-ad0fd73fea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:00.399770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879038f-8b2b-49ed-8937-ad0fd73fea07 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.400110 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879038f-8b2b-49ed-8937-ad0fd73fea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.400397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879038f-8b2b-49ed-8937-ad0fd73fea07 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.401114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879038f-8b2b-49ed-8937-ad0fd73fea07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:00.403216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879038f-8b2b-49ed-8937-ad0fd73fea07 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:00.425060 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-eeef7208-b723-4525-a629-b54e82d2b1e7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:00.426404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-eeef7208-b723-4525-a629-b54e82d2b1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:00.438021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 909/1815] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:00 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:00.443022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879038f-8b2b-49ed-8937-ad0fd73fea07 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:00.444961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4879038f-8b2b-49ed-8937-ad0fd73fea07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:00.448554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 907/1816] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:00.456226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d893a8-0c04-4c78-9e68-f2809bb9f133 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.456524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d893a8-0c04-4c78-9e68-f2809bb9f133 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.456805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d893a8-0c04-4c78-9e68-f2809bb9f133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.467730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879038f-8b2b-49ed-8937-ad0fd73fea07 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.474161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730252c8-e51b-4368-8852-638b9209fbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:00.474862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730252c8-e51b-4368-8852-638b9209fbe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.475153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730252c8-e51b-4368-8852-638b9209fbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.475433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730252c8-e51b-4368-8852-638b9209fbe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.476426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730252c8-e51b-4368-8852-638b9209fbe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:00.478330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730252c8-e51b-4368-8852-638b9209fbe2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:00.491371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 910/1817] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:00.492962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c9d893a8-0c04-4c78-9e68-f2809bb9f133 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:00.494220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c9d893a8-0c04-4c78-9e68-f2809bb9f133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:00.498457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730252c8-e51b-4368-8852-638b9209fbe2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:00.512460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730252c8-e51b-4368-8852-638b9209fbe2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.517321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 908/1818] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:00.520102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bd7e9-898f-4ee3-b3db-cdd2eb3f707b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:00.521314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bd7e9-898f-4ee3-b3db-cdd2eb3f707b admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.521465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bd7e9-898f-4ee3-b3db-cdd2eb3f707b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.521558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d893a8-0c04-4c78-9e68-f2809bb9f133 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:00.521744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bd7e9-898f-4ee3-b3db-cdd2eb3f707b admin admin] PATH_INFO: `/v3/projects/c4d3b9af73ff4950bf69e38e7e6c2bb9/users/b1965431e35c41b1bb65dd4399fda563/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.541999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bd7e9-898f-4ee3-b3db-cdd2eb3f707b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d3b9af73ff4950bf69e38e7e6c2bb9, user_id=b1965431e35c41b1bb65dd4399fda563, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:00.546470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d893a8-0c04-4c78-9e68-f2809bb9f133 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.561280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ea3dd-c7c4-4166-8501-b0f467b1532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:00.562156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bd7e9-898f-4ee3-b3db-cdd2eb3f707b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:00.562264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ea3dd-c7c4-4166-8501-b0f467b1532b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.562264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ea3dd-c7c4-4166-8501-b0f467b1532b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.562540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ea3dd-c7c4-4166-8501-b0f467b1532b admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.563375 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ea3dd-c7c4-4166-8501-b0f467b1532b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:00.563904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ea3dd-c7c4-4166-8501-b0f467b1532b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:00.565658 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ea3dd-c7c4-4166-8501-b0f467b1532b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:00.595173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 911/1819] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:00 2022] PUT /identity/v3/projects/c4d3b9af73ff4950bf69e38e7e6c2bb9/users/b1965431e35c41b1bb65dd4399fda563/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:00.596126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bd7e9-898f-4ee3-b3db-cdd2eb3f707b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:00.596358 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 909/1820] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:42:00.600831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ea3dd-c7c4-4166-8501-b0f467b1532b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:00.623579 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ea3dd-c7c4-4166-8501-b0f467b1532b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.767198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531c4a6-defe-4279-9f9d-99c4987aacd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:00.768070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531c4a6-defe-4279-9f9d-99c4987aacd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.768363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531c4a6-defe-4279-9f9d-99c4987aacd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.768644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531c4a6-defe-4279-9f9d-99c4987aacd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.769717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531c4a6-defe-4279-9f9d-99c4987aacd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:00.773015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531c4a6-defe-4279-9f9d-99c4987aacd2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:00.783319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bd7e9-898f-4ee3-b3db-cdd2eb3f707b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.800808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f14ad-30d4-47ef-9548-3d919fe772c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:00.801860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f14ad-30d4-47ef-9548-3d919fe772c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.802267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f14ad-30d4-47ef-9548-3d919fe772c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.802698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f14ad-30d4-47ef-9548-3d919fe772c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.803581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f14ad-30d4-47ef-9548-3d919fe772c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:00.805351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f14ad-30d4-47ef-9548-3d919fe772c3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:00.819964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 912/1821] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:00.824537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f14ad-30d4-47ef-9548-3d919fe772c3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:00.827554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 910/1822] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:00 2022] POST /identity/v3/projects => generated 369 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:00.830811 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531c4a6-defe-4279-9f9d-99c4987aacd2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:00.851220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f14ad-30d4-47ef-9548-3d919fe772c3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.913751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:00.960553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531c4a6-defe-4279-9f9d-99c4987aacd2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:00.973164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6a258-6878-489b-b0e1-adb5fe405bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:00.973875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6a258-6878-489b-b0e1-adb5fe405bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:00.974162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6a258-6878-489b-b0e1-adb5fe405bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:00.974439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6a258-6878-489b-b0e1-adb5fe405bb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:00.975149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6a258-6878-489b-b0e1-adb5fe405bb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:00.984130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6a258-6878-489b-b0e1-adb5fe405bb2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:00.999612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add996c7-dfa4-41bb-9161-7757dc0925e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:01.000538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add996c7-dfa4-41bb-9161-7757dc0925e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:01.001025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add996c7-dfa4-41bb-9161-7757dc0925e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:01.001311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add996c7-dfa4-41bb-9161-7757dc0925e9 admin admin] PATH_INFO: `/v3/projects/c4d3b9af73ff4950bf69e38e7e6c2bb9/users/b1965431e35c41b1bb65dd4399fda563/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:01.003744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add996c7-dfa4-41bb-9161-7757dc0925e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d3b9af73ff4950bf69e38e7e6c2bb9, user_id=b1965431e35c41b1bb65dd4399fda563, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:01.007980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add996c7-dfa4-41bb-9161-7757dc0925e9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:01.026556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 911/1823] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:00 2022] POST /identity/v3/users => generated 285 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:01.027576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6a258-6878-489b-b0e1-adb5fe405bb2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:01.035114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-93f6a258-6878-489b-b0e1-adb5fe405bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:01.054700 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 913/1824] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:00 2022] PUT /identity/v3/projects/c4d3b9af73ff4950bf69e38e7e6c2bb9/users/b1965431e35c41b1bb65dd4399fda563/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:01.055972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add996c7-dfa4-41bb-9161-7757dc0925e9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:01.069734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6a258-6878-489b-b0e1-adb5fe405bb2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:01.074865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:01.194419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add996c7-dfa4-41bb-9161-7757dc0925e9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:01.211230 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:01.212532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:01.213201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:01.214792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] PATH_INFO: `/v3/projects/e19722eed8d64c86b48c84fc08e41313` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:01.216745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19722eed8d64c86b48c84fc08e41313)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:01.217441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840e110-f409-4e94-9273-f5f58ae647e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:01.218187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840e110-f409-4e94-9273-f5f58ae647e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:01.218479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840e110-f409-4e94-9273-f5f58ae647e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:01.219421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840e110-f409-4e94-9273-f5f58ae647e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:01.220297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:01.220422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840e110-f409-4e94-9273-f5f58ae647e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:01.222095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840e110-f409-4e94-9273-f5f58ae647e5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:01.245147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 914/1825] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:01.246635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b60f9-a0be-4d51-a7d9-1fe5d29a8162 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:01.246933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b60f9-a0be-4d51-a7d9-1fe5d29a8162 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:01.247215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b60f9-a0be-4d51-a7d9-1fe5d29a8162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:01.321531 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2e4b60f9-a0be-4d51-a7d9-1fe5d29a8162 None None] MFA Rules not processed for user `b1965431e35c41b1bb65dd4399fda563`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:01.322992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2e4b60f9-a0be-4d51-a7d9-1fe5d29a8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:01.368936 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19722eed8d64c86b48c84fc08e41313', 'request_id': 'req-2a8a505e-e248-4a10-88d2-f581d0df58ce'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.383483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19722eed8d64c86b48c84fc08e41313', 'request_id': 'req-2a8a505e-e248-4a10-88d2-f581d0df58ce'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.391592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19722eed8d64c86b48c84fc08e41313', 'request_id': 'req-2a8a505e-e248-4a10-88d2-f581d0df58ce'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.446155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] The token cache is being invalidate because project e19722eed8d64c86b48c84fc08e41313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:01.447166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.447991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.465580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 912/1826] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:01 2022] DELETE /identity/v3/projects/e19722eed8d64c86b48c84fc08e41313 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:01.466647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:01.471767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:01.529841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 915/1827] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:01.531141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b60f9-a0be-4d51-a7d9-1fe5d29a8162 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:01.544776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2e4b60f9-a0be-4d51-a7d9-1fe5d29a8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:01.638371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:01.646128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a505e-e248-4a10-88d2-f581d0df58ce admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:01.676094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17e26-1fde-4f77-af94-a840ca2dc08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:01.676805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17e26-1fde-4f77-af94-a840ca2dc08d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:01.677090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17e26-1fde-4f77-af94-a840ca2dc08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:01.677364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17e26-1fde-4f77-af94-a840ca2dc08d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:01.678107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c17e26-1fde-4f77-af94-a840ca2dc08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:01.679801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c17e26-1fde-4f77-af94-a840ca2dc08d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:01.710759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 913/1828] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:01.711738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17e26-1fde-4f77-af94-a840ca2dc08d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:01.724419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b6c17e26-1fde-4f77-af94-a840ca2dc08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:01.741592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b60f9-a0be-4d51-a7d9-1fe5d29a8162 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:01.769870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:01.770782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:01.771062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:01.771450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] PATH_INFO: `/v3/users/c036b3f6937f4ae0ab2ea1b307f779e9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:01.776947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c036b3f6937f4ae0ab2ea1b307f779e9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:01.778927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:01.842477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17e26-1fde-4f77-af94-a840ca2dc08d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:01.851851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e69e-13f3-4a0e-bc92-17e53d4c6909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:01.852577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e69e-13f3-4a0e-bc92-17e53d4c6909 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:01.852968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e69e-13f3-4a0e-bc92-17e53d4c6909 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:01.853250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e69e-13f3-4a0e-bc92-17e53d4c6909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:01.855712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:01.874464 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4e69e-13f3-4a0e-bc92-17e53d4c6909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:01.879325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c036b3f6937f4ae0ab2ea1b307f779e9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.880049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c036b3f6937f4ae0ab2ea1b307f779e9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.885546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4e69e-13f3-4a0e-bc92-17e53d4c6909 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:01.902892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 914/1829] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:01.903661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e69e-13f3-4a0e-bc92-17e53d4c6909 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:01.904992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c036b3f6937f4ae0ab2ea1b307f779e9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.925246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c036b3f6937f4ae0ab2ea1b307f779e9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.927100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f251706c-dcde-4c23-82ef-c0b2e386e101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c036b3f6937f4ae0ab2ea1b307f779e9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:01.936935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e69e-13f3-4a0e-bc92-17e53d4c6909 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:01.953601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 916/1830] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:01 2022] DELETE /identity/v3/users/c036b3f6937f4ae0ab2ea1b307f779e9 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:01.961057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1d453-2ec8-4a7a-9f66-c34e00b73a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:01.966327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1d453-2ec8-4a7a-9f66-c34e00b73a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:01.966327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1d453-2ec8-4a7a-9f66-c34e00b73a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:01.966779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1d453-2ec8-4a7a-9f66-c34e00b73a37 admin admin] PATH_INFO: `/v3/projects/84a900bbc61d458ababa1989b99e9eeb/users/0a46545206604ab6b153c1f768f80e13/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:02.014814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1d453-2ec8-4a7a-9f66-c34e00b73a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a900bbc61d458ababa1989b99e9eeb, user_id=0a46545206604ab6b153c1f768f80e13, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:02.024013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1d453-2ec8-4a7a-9f66-c34e00b73a37 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:02.053040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 915/1831] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:01 2022] PUT /identity/v3/projects/84a900bbc61d458ababa1989b99e9eeb/users/0a46545206604ab6b153c1f768f80e13/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:02.065529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c595f5b-4ec7-4e9a-9bc2-6536119ba736 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:02.065905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c595f5b-4ec7-4e9a-9bc2-6536119ba736 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:02.066311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c595f5b-4ec7-4e9a-9bc2-6536119ba736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:02.090026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1d453-2ec8-4a7a-9f66-c34e00b73a37 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:02.091910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-68d1d453-2ec8-4a7a-9f66-c34e00b73a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:02.151737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0c595f5b-4ec7-4e9a-9bc2-6536119ba736 None None] MFA Rules not processed for user `0a46545206604ab6b153c1f768f80e13`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:02.153170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0c595f5b-4ec7-4e9a-9bc2-6536119ba736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:02.229624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1d453-2ec8-4a7a-9f66-c34e00b73a37 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:02.240314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c329898-9aa5-462c-bff8-7ed67f380f30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:02.244249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c329898-9aa5-462c-bff8-7ed67f380f30 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:02.244523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c329898-9aa5-462c-bff8-7ed67f380f30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:02.244747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c329898-9aa5-462c-bff8-7ed67f380f30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:02.256698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0c329898-9aa5-462c-bff8-7ed67f380f30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:02.350211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 917/1832] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:02.353476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c595f5b-4ec7-4e9a-9bc2-6536119ba736 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:02.357252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0c595f5b-4ec7-4e9a-9bc2-6536119ba736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:02.451995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c329898-9aa5-462c-bff8-7ed67f380f30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:02.457518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c329898-9aa5-462c-bff8-7ed67f380f30 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:02.474621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 916/1833] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:02.566926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c595f5b-4ec7-4e9a-9bc2-6536119ba736 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:02.596366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653c03-cfc5-4989-ac0c-fcf8d7a49baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:02.597060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653c03-cfc5-4989-ac0c-fcf8d7a49baa admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:02.597340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653c03-cfc5-4989-ac0c-fcf8d7a49baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:02.597612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653c03-cfc5-4989-ac0c-fcf8d7a49baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:02.598309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6653c03-cfc5-4989-ac0c-fcf8d7a49baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:02.600264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6653c03-cfc5-4989-ac0c-fcf8d7a49baa admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:02.711315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 918/1834] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:02 2022] POST /identity/v3/projects => generated 367 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:02.715303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c329898-9aa5-462c-bff8-7ed67f380f30 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:02.777605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c329898-9aa5-462c-bff8-7ed67f380f30 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:02.851580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:02.897731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653c03-cfc5-4989-ac0c-fcf8d7a49baa admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:02.899697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f6653c03-cfc5-4989-ac0c-fcf8d7a49baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:02.999586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:03.006721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2d36-2d67-435d-be04-b9d602fc8e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:03.007515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2d36-2d67-435d-be04-b9d602fc8e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.008242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2d36-2d67-435d-be04-b9d602fc8e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.017077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2d36-2d67-435d-be04-b9d602fc8e92 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:03.017885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b2d36-2d67-435d-be04-b9d602fc8e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:03.026432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b2d36-2d67-435d-be04-b9d602fc8e92 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:03.098749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 917/1835] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:02 2022] POST /identity/v3/users => generated 290 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:03.102689 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2d36-2d67-435d-be04-b9d602fc8e92 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:03.115764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2d36-2d67-435d-be04-b9d602fc8e92 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:03.124875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ba9b2-10bc-4e21-b9c4-b799097eb330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:03.125710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ba9b2-10bc-4e21-b9c4-b799097eb330 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.126672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ba9b2-10bc-4e21-b9c4-b799097eb330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.127275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ba9b2-10bc-4e21-b9c4-b799097eb330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:03.128824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ba9b2-10bc-4e21-b9c4-b799097eb330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:03.132927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ba9b2-10bc-4e21-b9c4-b799097eb330 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:03.151120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653c03-cfc5-4989-ac0c-fcf8d7a49baa admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:03.162430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 918/1836] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:03.164368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:03.164579 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ba9b2-10bc-4e21-b9c4-b799097eb330 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:03.166987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.167283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.167708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] PATH_INFO: `/v3/projects/98f0a7262799432694e6578838d558f6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:03.169430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-444ba9b2-10bc-4e21-b9c4-b799097eb330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:03.171366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f0a7262799432694e6578838d558f6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:03.176849 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:03.197569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ba9b2-10bc-4e21-b9c4-b799097eb330 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:03.209515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:03.210528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.210821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.211093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] PATH_INFO: `/v3/users/2e7e652a39bd411b8dcdcaf87b7e635d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:03.220514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7e652a39bd411b8dcdcaf87b7e635d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:03.227871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:03.299851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f0a7262799432694e6578838d558f6', 'request_id': 'req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.308776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f0a7262799432694e6578838d558f6', 'request_id': 'req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.317485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f0a7262799432694e6578838d558f6', 'request_id': 'req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.355234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] The token cache is being invalidate because project 98f0a7262799432694e6578838d558f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:03.355571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.356581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.363010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 919/1837] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:02 2022] DELETE /identity/v3/projects/98f0a7262799432694e6578838d558f6 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:03.369199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:03.370576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7e652a39bd411b8dcdcaf87b7e635d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.370712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:03.371531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7e652a39bd411b8dcdcaf87b7e635d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.401787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7e652a39bd411b8dcdcaf87b7e635d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.413495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7e652a39bd411b8dcdcaf87b7e635d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.431031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7e652a39bd411b8dcdcaf87b7e635d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.439499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 919/1838] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:03 2022] DELETE /identity/v3/users/2e7e652a39bd411b8dcdcaf87b7e635d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:03.460044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:03.462508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:03.491085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72016e-fc4f-4d99-8ea6-6c6afdc96c71 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:03.577302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47ba4c-97c0-480c-baa1-18db27f3f59b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:03.583545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:03.583801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f5013-ce3a-424b-b25a-03cc29a89afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:03.584464 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.584751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.585615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] PATH_INFO: `/v3/users/f7224be8b49a414987898221eacb2283` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:03.586596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5013-ce3a-424b-b25a-03cc29a89afd admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.586987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7224be8b49a414987898221eacb2283)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:03.587280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5013-ce3a-424b-b25a-03cc29a89afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.587761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5013-ce3a-424b-b25a-03cc29a89afd admin admin] PATH_INFO: `/v3/projects/84db565a067c4ec9b76389391f1ecad0/users/ba80cef3b341431ca66139706d148786/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:03.591870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:03.595178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:03.613334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f5013-ce3a-424b-b25a-03cc29a89afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=84db565a067c4ec9b76389391f1ecad0, user_id=ba80cef3b341431ca66139706d148786, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:03.617846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f5013-ce3a-424b-b25a-03cc29a89afd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:03.636773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:03.649703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 920/1839] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:03 2022] PUT /identity/v3/projects/84db565a067c4ec9b76389391f1ecad0/users/ba80cef3b341431ca66139706d148786/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:03.651331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823e8d7-a04b-49c3-8b21-8a90d35f31a3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.651728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823e8d7-a04b-49c3-8b21-8a90d35f31a3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.652373 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823e8d7-a04b-49c3-8b21-8a90d35f31a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:03.683200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0823e8d7-a04b-49c3-8b21-8a90d35f31a3 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:03.683421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7224be8b49a414987898221eacb2283'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.684319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7224be8b49a414987898221eacb2283'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.684704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0823e8d7-a04b-49c3-8b21-8a90d35f31a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:03.715687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7224be8b49a414987898221eacb2283'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.724408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7224be8b49a414987898221eacb2283'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.738345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7224be8b49a414987898221eacb2283'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:03.751975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 920/1840] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:03 2022] DELETE /identity/v3/users/f7224be8b49a414987898221eacb2283 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:03.753039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:03.755095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:03.772760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 921/1841] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:03.793440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823e8d7-a04b-49c3-8b21-8a90d35f31a3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:03.795143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0823e8d7-a04b-49c3-8b21-8a90d35f31a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:03.835419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0317c8-46d0-4b16-988e-7cea50387898 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:03.860190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad78c8-6ce5-45b8-afd1-605620ca4dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:03.860190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad78c8-6ce5-45b8-afd1-605620ca4dff admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.860190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad78c8-6ce5-45b8-afd1-605620ca4dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.860190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad78c8-6ce5-45b8-afd1-605620ca4dff admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:03.860613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad78c8-6ce5-45b8-afd1-605620ca4dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:03.864915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad78c8-6ce5-45b8-afd1-605620ca4dff admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:03.895893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 921/1842] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:03.899173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad78c8-6ce5-45b8-afd1-605620ca4dff admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:03.920358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823e8d7-a04b-49c3-8b21-8a90d35f31a3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:03.925337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad78c8-6ce5-45b8-afd1-605620ca4dff admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:03.936012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7177e-3ade-4cb0-9494-ce029a3c5720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:03.941479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7177e-3ade-4cb0-9494-ce029a3c5720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.941479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7177e-3ade-4cb0-9494-ce029a3c5720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.941703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7177e-3ade-4cb0-9494-ce029a3c5720 admin admin] PATH_INFO: `/v3/projects/84db565a067c4ec9b76389391f1ecad0/users/ba80cef3b341431ca66139706d148786/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:03.948512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7177e-3ade-4cb0-9494-ce029a3c5720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84db565a067c4ec9b76389391f1ecad0, user_id=ba80cef3b341431ca66139706d148786, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:03.954036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7177e-3ade-4cb0-9494-ce029a3c5720 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:03.979491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 922/1843] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:03 2022] PUT /identity/v3/projects/84db565a067c4ec9b76389391f1ecad0/users/ba80cef3b341431ca66139706d148786/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:03.987549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bedd6-bb77-461d-bb5e-b5f970053010 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:03.987850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bedd6-bb77-461d-bb5e-b5f970053010 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:03.988721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bedd6-bb77-461d-bb5e-b5f970053010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:04.031165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c96bedd6-bb77-461d-bb5e-b5f970053010 None None] MFA Rules not processed for user `ba80cef3b341431ca66139706d148786`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:04.032653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c96bedd6-bb77-461d-bb5e-b5f970053010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:04.080064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ffdc8-7391-4b5f-8b61-6755861f12eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:04.080725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ffdc8-7391-4b5f-8b61-6755861f12eb service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:04.081014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ffdc8-7391-4b5f-8b61-6755861f12eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:04.081366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ffdc8-7391-4b5f-8b61-6755861f12eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:04.095606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ffdc8-7391-4b5f-8b61-6755861f12eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:04.098273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ffdc8-7391-4b5f-8b61-6755861f12eb service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:04.117213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 922/1844] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:04.178921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 923/1845] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:04.190472 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ffdc8-7391-4b5f-8b61-6755861f12eb service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:04.191948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dd5ffdc8-7391-4b5f-8b61-6755861f12eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:04.274486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bedd6-bb77-461d-bb5e-b5f970053010 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:04.276349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c96bedd6-bb77-461d-bb5e-b5f970053010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:04.287751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:04.391204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:04.416725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ffdc8-7391-4b5f-8b61-6755861f12eb service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:04.439139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bedd6-bb77-461d-bb5e-b5f970053010 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:04.455621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:04.456680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:04.456975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:04.457259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] PATH_INFO: `/v3/projects/a18edc9e4ea84bd08d9202bc292d29b1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:04.458630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18edc9e4ea84bd08d9202bc292d29b1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:04.462289 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:04.511150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f4c90-a63a-40a8-ab68-2a35c028ba50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:04.512687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f4c90-a63a-40a8-ab68-2a35c028ba50 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:04.513513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f4c90-a63a-40a8-ab68-2a35c028ba50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:04.513513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f4c90-a63a-40a8-ab68-2a35c028ba50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:04.525594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f4c90-a63a-40a8-ab68-2a35c028ba50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:04.528269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f4c90-a63a-40a8-ab68-2a35c028ba50 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:04.542114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 923/1846] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:04.543760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c8fdd-232d-4dae-afb9-da561f01641b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:04.544137 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c8fdd-232d-4dae-afb9-da561f01641b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:04.544616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c8fdd-232d-4dae-afb9-da561f01641b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:04.546355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 924/1847] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:42:04.559044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18edc9e4ea84bd08d9202bc292d29b1', 'request_id': 'req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:04.569458 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18edc9e4ea84bd08d9202bc292d29b1', 'request_id': 'req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:04.577701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18edc9e4ea84bd08d9202bc292d29b1', 'request_id': 'req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:04.592047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] The token cache is being invalidate because project a18edc9e4ea84bd08d9202bc292d29b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:04.592414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:04.593338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6b1b1ea8-4218-49ec-8765-82b04ceecdd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:04.596325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 924/1848] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:04 2022] DELETE /identity/v3/projects/a18edc9e4ea84bd08d9202bc292d29b1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:04.602597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78f17a-d051-491b-a01b-402d13fdea00 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:04.603101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78f17a-d051-491b-a01b-402d13fdea00 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:04.603101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78f17a-d051-491b-a01b-402d13fdea00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:04.607710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abb256-752d-48aa-baa9-036bb8a13be7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:04.608594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abb256-752d-48aa-baa9-036bb8a13be7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:04.608999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abb256-752d-48aa-baa9-036bb8a13be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:04.673659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-60abb256-752d-48aa-baa9-036bb8a13be7 None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:04.674556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2f78f17a-d051-491b-a01b-402d13fdea00 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:04.678866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-60abb256-752d-48aa-baa9-036bb8a13be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:04.680398 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2f78f17a-d051-491b-a01b-402d13fdea00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:04.935162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 925/1849] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:04.940174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 925/1850] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:42:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:04.963186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee57d89-4c95-44f8-8936-79209fc5fb40 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:04.967314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee57d89-4c95-44f8-8936-79209fc5fb40 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:04.967969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee57d89-4c95-44f8-8936-79209fc5fb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:05.032121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cee57d89-4c95-44f8-8936-79209fc5fb40 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:05.033949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cee57d89-4c95-44f8-8936-79209fc5fb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:05.044437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 926/1851] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:05.060381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abb256-752d-48aa-baa9-036bb8a13be7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:05.093003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee57d89-4c95-44f8-8936-79209fc5fb40 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:05.093003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cee57d89-4c95-44f8-8936-79209fc5fb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:05.108322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abb256-752d-48aa-baa9-036bb8a13be7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:05.135066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071085c-f5fe-442b-961f-181da8eb8e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:05.136729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee57d89-4c95-44f8-8936-79209fc5fb40 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:05.136949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071085c-f5fe-442b-961f-181da8eb8e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:05.137340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071085c-f5fe-442b-961f-181da8eb8e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:05.137933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071085c-f5fe-442b-961f-181da8eb8e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:05.141698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071085c-f5fe-442b-961f-181da8eb8e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:05.142146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071085c-f5fe-442b-961f-181da8eb8e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:05.145355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071085c-f5fe-442b-961f-181da8eb8e8a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:05.147422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:05.148520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:05.148812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:05.149090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] PATH_INFO: `/v3/projects/ba92d95056ca4122b7462696ce2b43bb` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:05.150713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba92d95056ca4122b7462696ce2b43bb)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:05.152760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:05.173271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 926/1852] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:42:05.177582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071085c-f5fe-442b-961f-181da8eb8e8a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:05.193693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071085c-f5fe-442b-961f-181da8eb8e8a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:05.201668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d10ca6-0f49-4c1c-a9cf-60eea881cb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:05.202797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d10ca6-0f49-4c1c-a9cf-60eea881cb8d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:05.203957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d10ca6-0f49-4c1c-a9cf-60eea881cb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:05.203957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d10ca6-0f49-4c1c-a9cf-60eea881cb8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:05.204268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d10ca6-0f49-4c1c-a9cf-60eea881cb8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:05.206316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d10ca6-0f49-4c1c-a9cf-60eea881cb8d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:05.221509 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:05.235629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba92d95056ca4122b7462696ce2b43bb', 'request_id': 'req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.243041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba92d95056ca4122b7462696ce2b43bb', 'request_id': 'req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.257686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba92d95056ca4122b7462696ce2b43bb', 'request_id': 'req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.268442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 927/1853] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:05 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:05.273781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d10ca6-0f49-4c1c-a9cf-60eea881cb8d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:05.312627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] The token cache is being invalidate because project ba92d95056ca4122b7462696ce2b43bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:05.313549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.314737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.318436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 927/1854] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:05 2022] DELETE /identity/v3/projects/ba92d95056ca4122b7462696ce2b43bb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:05.324014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:05.328191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:05.381359 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:05.426817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d10ca6-0f49-4c1c-a9cf-60eea881cb8d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:05.439383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a5de-6dac-4f5f-a387-f0907c897c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:05.440483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a5de-6dac-4f5f-a387-f0907c897c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:05.441836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a5de-6dac-4f5f-a387-f0907c897c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:05.441836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a5de-6dac-4f5f-a387-f0907c897c88 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:05.444064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32a5de-6dac-4f5f-a387-f0907c897c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:05.450159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32a5de-6dac-4f5f-a387-f0907c897c88 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:05.497461 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 928/1855] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:05 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:05.499729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a5de-6dac-4f5f-a387-f0907c897c88 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:05.499881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc25e-bfe0-4207-bb0a-7009b4c5eee4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:05.502218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fb32a5de-6dac-4f5f-a387-f0907c897c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:05.524262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a5de-6dac-4f5f-a387-f0907c897c88 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:05.531054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:05.532097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:05.532449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:05.534068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] PATH_INFO: `/v3/users/b1965431e35c41b1bb65dd4399fda563` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:05.535572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1965431e35c41b1bb65dd4399fda563)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:05.541099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:05.611724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:05.618441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:05.618744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:05.619050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:05.621163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:05.627682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1965431e35c41b1bb65dd4399fda563'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.628478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1965431e35c41b1bb65dd4399fda563'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.637243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:05.639052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:05.658196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1965431e35c41b1bb65dd4399fda563'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.668574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1965431e35c41b1bb65dd4399fda563'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.680641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1965431e35c41b1bb65dd4399fda563'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:05.680890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 928/1856] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:05.682010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:05.688250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:05.692204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 929/1857] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:05 2022] DELETE /identity/v3/users/b1965431e35c41b1bb65dd4399fda563 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:05.747749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b745256-5590-45d2-974c-8b220fe214e1 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:05.951496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fa7fb-ad20-4a0b-88f6-4290c83078c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:05.952214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fa7fb-ad20-4a0b-88f6-4290c83078c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:05.952878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fa7fb-ad20-4a0b-88f6-4290c83078c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:05.952878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fa7fb-ad20-4a0b-88f6-4290c83078c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:05.953514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fa7fb-ad20-4a0b-88f6-4290c83078c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:05.955204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fa7fb-ad20-4a0b-88f6-4290c83078c8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:05.984504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 929/1858] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:05.988968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:05.992945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:06.015918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a75a9-690e-4025-b5c3-0ddceb2cdc45 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:06.022268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0210276-42ae-4d22-86ae-790ae1c79f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:06.023728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0210276-42ae-4d22-86ae-790ae1c79f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:06.024938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0210276-42ae-4d22-86ae-790ae1c79f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:06.025224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0210276-42ae-4d22-86ae-790ae1c79f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:06.027529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0210276-42ae-4d22-86ae-790ae1c79f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:06.029511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0210276-42ae-4d22-86ae-790ae1c79f20 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:06.047981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 930/1859] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:06.052072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fa7fb-ad20-4a0b-88f6-4290c83078c8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:06.075365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fa7fb-ad20-4a0b-88f6-4290c83078c8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:06.104941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b3efe-c9f5-436a-b460-93ad1d251445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:06.105887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b3efe-c9f5-436a-b460-93ad1d251445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:06.106174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b3efe-c9f5-436a-b460-93ad1d251445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:06.106456 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b3efe-c9f5-436a-b460-93ad1d251445 admin admin] PATH_INFO: `/v3/projects/510401c9ecba4cf9990a3cc7d6d963ad/users/f12d7ce7904e41b79f3b3e9ecb086700/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:06.190328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b3efe-c9f5-436a-b460-93ad1d251445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510401c9ecba4cf9990a3cc7d6d963ad, user_id=f12d7ce7904e41b79f3b3e9ecb086700, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:06.194943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b3efe-c9f5-436a-b460-93ad1d251445 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:06.223367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 930/1860] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:06 2022] PUT /identity/v3/projects/510401c9ecba4cf9990a3cc7d6d963ad/users/f12d7ce7904e41b79f3b3e9ecb086700/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:06.238159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04a60-7d28-4166-a1a6-965bf9fb1344 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:06.238457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04a60-7d28-4166-a1a6-965bf9fb1344 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:06.238783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04a60-7d28-4166-a1a6-965bf9fb1344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:06.274648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-95d04a60-7d28-4166-a1a6-965bf9fb1344 None None] MFA Rules not processed for user `f12d7ce7904e41b79f3b3e9ecb086700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:06.276727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-95d04a60-7d28-4166-a1a6-965bf9fb1344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:06.355832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:06.419739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 931/1861] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:06.425381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b3efe-c9f5-436a-b460-93ad1d251445 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:06.447872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:06.534121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b3efe-c9f5-436a-b460-93ad1d251445 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:06.540345 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328d1a3-c0d1-40a2-b0de-6f950fd6fb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:06.541029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328d1a3-c0d1-40a2-b0de-6f950fd6fb5e admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:06.541480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328d1a3-c0d1-40a2-b0de-6f950fd6fb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:06.541995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328d1a3-c0d1-40a2-b0de-6f950fd6fb5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:06.543133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328d1a3-c0d1-40a2-b0de-6f950fd6fb5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:06.548247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328d1a3-c0d1-40a2-b0de-6f950fd6fb5e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:06.615817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 931/1862] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:06 2022] POST /identity/v3/projects => generated 363 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:06.620225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04a60-7d28-4166-a1a6-965bf9fb1344 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:06.633641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04a60-7d28-4166-a1a6-965bf9fb1344 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:06.707332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb6819-b3b1-4b1d-a9c4-227f0f0eb8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:06.708073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb6819-b3b1-4b1d-a9c4-227f0f0eb8db admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:06.708586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb6819-b3b1-4b1d-a9c4-227f0f0eb8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:06.708939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb6819-b3b1-4b1d-a9c4-227f0f0eb8db admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:06.709714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb6819-b3b1-4b1d-a9c4-227f0f0eb8db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:06.711875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb6819-b3b1-4b1d-a9c4-227f0f0eb8db admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:06.744030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328d1a3-c0d1-40a2-b0de-6f950fd6fb5e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:06.754135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 932/1863] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:06 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:06.758909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328d1a3-c0d1-40a2-b0de-6f950fd6fb5e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:06.765840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e808119-fc85-4b94-968e-bf932368baba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:06.766950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e808119-fc85-4b94-968e-bf932368baba admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:06.767443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e808119-fc85-4b94-968e-bf932368baba admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:06.767955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e808119-fc85-4b94-968e-bf932368baba admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:06.769862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e808119-fc85-4b94-968e-bf932368baba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:06.772018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e808119-fc85-4b94-968e-bf932368baba admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:06.790469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 932/1864] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:06.798581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb6819-b3b1-4b1d-a9c4-227f0f0eb8db admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:06.811269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb6819-b3b1-4b1d-a9c4-227f0f0eb8db admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:06.817168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fba191-16d6-4826-8d94-b3060530b240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:06.818496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fba191-16d6-4826-8d94-b3060530b240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:06.818836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fba191-16d6-4826-8d94-b3060530b240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:06.819801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fba191-16d6-4826-8d94-b3060530b240 admin admin] PATH_INFO: `/v3/projects/9b0d59c49c384d8f90e0f28f7ec406c6/users/1529aafd9a184bfab3a444ecfdf1cfc9/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:06.845722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fba191-16d6-4826-8d94-b3060530b240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0d59c49c384d8f90e0f28f7ec406c6, user_id=1529aafd9a184bfab3a444ecfdf1cfc9, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:06.849081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fba191-16d6-4826-8d94-b3060530b240 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:06.874311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 933/1865] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:06 2022] PUT /identity/v3/projects/9b0d59c49c384d8f90e0f28f7ec406c6/users/1529aafd9a184bfab3a444ecfdf1cfc9/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:06.878647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e808119-fc85-4b94-968e-bf932368baba admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:06.892300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e808119-fc85-4b94-968e-bf932368baba admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:07.047530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fba191-16d6-4826-8d94-b3060530b240 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:07.049401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-89fba191-16d6-4826-8d94-b3060530b240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:07.057502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16cd0d-8f1b-4978-80ad-216e5bb1b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:07.058197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16cd0d-8f1b-4978-80ad-216e5bb1b575 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.058485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16cd0d-8f1b-4978-80ad-216e5bb1b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.059062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16cd0d-8f1b-4978-80ad-216e5bb1b575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.059835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16cd0d-8f1b-4978-80ad-216e5bb1b575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:07.067168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16cd0d-8f1b-4978-80ad-216e5bb1b575 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:07.074837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fba191-16d6-4826-8d94-b3060530b240 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:07.083001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:07.084613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.084905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.085187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] PATH_INFO: `/v3/users/0a46545206604ab6b153c1f768f80e13` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.086482 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a46545206604ab6b153c1f768f80e13)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:07.090605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 933/1866] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:07.090768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:07.092469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16cd0d-8f1b-4978-80ad-216e5bb1b575 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:07.104170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16cd0d-8f1b-4978-80ad-216e5bb1b575 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:07.112287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9f600-6060-4064-b47f-652d01267998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:07.115624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9f600-6060-4064-b47f-652d01267998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.115923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9f600-6060-4064-b47f-652d01267998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.116214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9f600-6060-4064-b47f-652d01267998 admin admin] PATH_INFO: `/v3/projects/9b0d59c49c384d8f90e0f28f7ec406c6/users/1529aafd9a184bfab3a444ecfdf1cfc9/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.119757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9f600-6060-4064-b47f-652d01267998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0d59c49c384d8f90e0f28f7ec406c6, user_id=1529aafd9a184bfab3a444ecfdf1cfc9, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:07.124663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9f600-6060-4064-b47f-652d01267998 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:07.145884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 934/1867] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:07 2022] PUT /identity/v3/projects/9b0d59c49c384d8f90e0f28f7ec406c6/users/1529aafd9a184bfab3a444ecfdf1cfc9/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:07.156435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48748f99-6206-4bcc-b48f-81184dd77106 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.156606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48748f99-6206-4bcc-b48f-81184dd77106 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.157031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48748f99-6206-4bcc-b48f-81184dd77106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.176766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a46545206604ab6b153c1f768f80e13'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.177531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a46545206604ab6b153c1f768f80e13'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.193564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a46545206604ab6b153c1f768f80e13'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.204257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a46545206604ab6b153c1f768f80e13'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.207488 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-48748f99-6206-4bcc-b48f-81184dd77106 None None] MFA Rules not processed for user `1529aafd9a184bfab3a444ecfdf1cfc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:07.209337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-48748f99-6206-4bcc-b48f-81184dd77106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:07.216494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a3dafecf-fd95-43a5-8314-7cc6c20bc601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a46545206604ab6b153c1f768f80e13'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.228617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 934/1868] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:07 2022] DELETE /identity/v3/users/0a46545206604ab6b153c1f768f80e13 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:07.263569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f4d73-c1eb-4e85-b55f-a07a85d58438 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.264014 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f4d73-c1eb-4e85-b55f-a07a85d58438 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.265383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f4d73-c1eb-4e85-b55f-a07a85d58438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.306865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e08f4d73-c1eb-4e85-b55f-a07a85d58438 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:07.310408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e08f4d73-c1eb-4e85-b55f-a07a85d58438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:07.316971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:07.365079 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:07.388545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 935/1869] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:07 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:07.389594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48748f99-6206-4bcc-b48f-81184dd77106 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:07.400770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-48748f99-6206-4bcc-b48f-81184dd77106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:07.433494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 935/1870] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:07.434424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f4d73-c1eb-4e85-b55f-a07a85d58438 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:07.436521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e08f4d73-c1eb-4e85-b55f-a07a85d58438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:07.522234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f4d73-c1eb-4e85-b55f-a07a85d58438 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:07.528860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c97fd-5800-41b4-92f2-90874aa98f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:07.529518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48748f99-6206-4bcc-b48f-81184dd77106 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:07.531154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c97fd-5800-41b4-92f2-90874aa98f90 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.531488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c97fd-5800-41b4-92f2-90874aa98f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.531785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c97fd-5800-41b4-92f2-90874aa98f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.536287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:07.537204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.537479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.537765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] PATH_INFO: `/v3/projects/c4d3b9af73ff4950bf69e38e7e6c2bb9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.544515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d3b9af73ff4950bf69e38e7e6c2bb9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:07.547303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:07.614048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d3b9af73ff4950bf69e38e7e6c2bb9', 'request_id': 'req-f7fd9ff4-a611-4648-9b62-21594c5c8012'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.645519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d3b9af73ff4950bf69e38e7e6c2bb9', 'request_id': 'req-f7fd9ff4-a611-4648-9b62-21594c5c8012'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.648240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c97fd-5800-41b4-92f2-90874aa98f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:07.650463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c97fd-5800-41b4-92f2-90874aa98f90 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:07.662522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d3b9af73ff4950bf69e38e7e6c2bb9', 'request_id': 'req-f7fd9ff4-a611-4648-9b62-21594c5c8012'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.665016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 936/1871] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:07 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:07.666233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c97fd-5800-41b4-92f2-90874aa98f90 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:07.668278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-687c97fd-5800-41b4-92f2-90874aa98f90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:07.708536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] The token cache is being invalidate because project c4d3b9af73ff4950bf69e38e7e6c2bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:07.709778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.711669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f7fd9ff4-a611-4648-9b62-21594c5c8012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:07.716090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 936/1872] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:07 2022] DELETE /identity/v3/projects/c4d3b9af73ff4950bf69e38e7e6c2bb9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:07.723868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3f500-a5e1-4fae-a08f-b81ab5bf1e3f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.724174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3f500-a5e1-4fae-a08f-b81ab5bf1e3f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.724449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3f500-a5e1-4fae-a08f-b81ab5bf1e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.762175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c97fd-5800-41b4-92f2-90874aa98f90 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:07.773366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-12f3f500-a5e1-4fae-a08f-b81ab5bf1e3f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:07.776950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-12f3f500-a5e1-4fae-a08f-b81ab5bf1e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:07.788943 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:07.851377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07192f-0431-4c2e-a960-511155375524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:07.852143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07192f-0431-4c2e-a960-511155375524 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.852610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07192f-0431-4c2e-a960-511155375524 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.852904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07192f-0431-4c2e-a960-511155375524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.854980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2e07192f-0431-4c2e-a960-511155375524 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:07.951392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 937/1873] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:07.952885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3f500-a5e1-4fae-a08f-b81ab5bf1e3f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:07.952994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07192f-0431-4c2e-a960-511155375524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:07.954674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-12f3f500-a5e1-4fae-a08f-b81ab5bf1e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:07.955314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07192f-0431-4c2e-a960-511155375524 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:07.984449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 937/1874] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:07.986121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33b43-6edf-434b-99b4-c7968f1373ab None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:07.986398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33b43-6edf-434b-99b4-c7968f1373ab None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:07.986678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33b43-6edf-434b-99b4-c7968f1373ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:07.998883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:08.027661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-08b33b43-6edf-434b-99b4-c7968f1373ab None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:08.031495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-08b33b43-6edf-434b-99b4-c7968f1373ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:08.039835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3f500-a5e1-4fae-a08f-b81ab5bf1e3f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.057374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386967e1-ee84-407d-91fd-86987f1e0721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.059119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 938/1875] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:08.060827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386967e1-ee84-407d-91fd-86987f1e0721 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.062858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386967e1-ee84-407d-91fd-86987f1e0721 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.063155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b33b43-6edf-434b-99b4-c7968f1373ab None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:08.066240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386967e1-ee84-407d-91fd-86987f1e0721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.069757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-386967e1-ee84-407d-91fd-86987f1e0721 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:08.077494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b33b43-6edf-434b-99b4-c7968f1373ab None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.088307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ed9fd-8282-4b67-875f-cfc08f4640eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.089140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ed9fd-8282-4b67-875f-cfc08f4640eb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.089452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ed9fd-8282-4b67-875f-cfc08f4640eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.089774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ed9fd-8282-4b67-875f-cfc08f4640eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.090569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ed9fd-8282-4b67-875f-cfc08f4640eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:08.091036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ed9fd-8282-4b67-875f-cfc08f4640eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.092926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ed9fd-8282-4b67-875f-cfc08f4640eb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.111499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 939/1876] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:42:08.116904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ed9fd-8282-4b67-875f-cfc08f4640eb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:08.128150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ed9fd-8282-4b67-875f-cfc08f4640eb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.134183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332c557-e7b6-485f-ab2d-419b8beec9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.134926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332c557-e7b6-485f-ab2d-419b8beec9dc admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.135225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332c557-e7b6-485f-ab2d-419b8beec9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.135949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332c557-e7b6-485f-ab2d-419b8beec9dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.136692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332c557-e7b6-485f-ab2d-419b8beec9dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.138714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332c557-e7b6-485f-ab2d-419b8beec9dc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.159719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386967e1-ee84-407d-91fd-86987f1e0721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.161428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386967e1-ee84-407d-91fd-86987f1e0721 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.172088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 938/1877] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:08.183562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 940/1878] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:08 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:08.184689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386967e1-ee84-407d-91fd-86987f1e0721 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:08.201427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386967e1-ee84-407d-91fd-86987f1e0721 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.281535 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687a900-0b09-4cd8-b2c4-0f2e4a60c028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.282436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a900-0b09-4cd8-b2c4-0f2e4a60c028 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.282817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a900-0b09-4cd8-b2c4-0f2e4a60c028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.283502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a900-0b09-4cd8-b2c4-0f2e4a60c028 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.284007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687a900-0b09-4cd8-b2c4-0f2e4a60c028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.286187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687a900-0b09-4cd8-b2c4-0f2e4a60c028 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.309971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 939/1879] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:08 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:08.314022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332c557-e7b6-485f-ab2d-419b8beec9dc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:08.327601 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332c557-e7b6-485f-ab2d-419b8beec9dc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.332676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5912b01-ec82-43e8-bd45-a36a976b10fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.333489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5912b01-ec82-43e8-bd45-a36a976b10fa admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.333682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5912b01-ec82-43e8-bd45-a36a976b10fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.333923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5912b01-ec82-43e8-bd45-a36a976b10fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.334663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5912b01-ec82-43e8-bd45-a36a976b10fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.336503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5912b01-ec82-43e8-bd45-a36a976b10fa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.348502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 941/1880] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:08.352377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687a900-0b09-4cd8-b2c4-0f2e4a60c028 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:08.363196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687a900-0b09-4cd8-b2c4-0f2e4a60c028 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.369266 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380bcc3-dbcc-470f-a11a-6cceef04e3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.372467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380bcc3-dbcc-470f-a11a-6cceef04e3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.372467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380bcc3-dbcc-470f-a11a-6cceef04e3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.372467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380bcc3-dbcc-470f-a11a-6cceef04e3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.372467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380bcc3-dbcc-470f-a11a-6cceef04e3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.373678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380bcc3-dbcc-470f-a11a-6cceef04e3c6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.385737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 940/1881] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:08.391037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5912b01-ec82-43e8-bd45-a36a976b10fa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:08.401856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5912b01-ec82-43e8-bd45-a36a976b10fa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.408355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9609bf7-d836-4870-a3c1-678196a23995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.409611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9609bf7-d836-4870-a3c1-678196a23995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.409611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9609bf7-d836-4870-a3c1-678196a23995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.409863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9609bf7-d836-4870-a3c1-678196a23995 admin admin] PATH_INFO: `/v3/projects/25ffcc0b6bcd4ba2b02faf9c538cc66c/users/cdfeb15aa05c4809aed9a899d9d2ee1f/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.428684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9609bf7-d836-4870-a3c1-678196a23995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ffcc0b6bcd4ba2b02faf9c538cc66c, user_id=cdfeb15aa05c4809aed9a899d9d2ee1f, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.431568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9609bf7-d836-4870-a3c1-678196a23995 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.456356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 942/1882] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:08 2022] PUT /identity/v3/projects/25ffcc0b6bcd4ba2b02faf9c538cc66c/users/cdfeb15aa05c4809aed9a899d9d2ee1f/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:08.458764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea213-ba4c-4f33-a571-471c5a7286dc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.459063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea213-ba4c-4f33-a571-471c5a7286dc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.459382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea213-ba4c-4f33-a571-471c5a7286dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.531701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-217ea213-ba4c-4f33-a571-471c5a7286dc None None] MFA Rules not processed for user `cdfeb15aa05c4809aed9a899d9d2ee1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:08.533143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-217ea213-ba4c-4f33-a571-471c5a7286dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:08.625807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:08.659663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9609bf7-d836-4870-a3c1-678196a23995 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:08.663503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 941/1883] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:08.722940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:08.738868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9609bf7-d836-4870-a3c1-678196a23995 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.744573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafc7c0-0966-4596-8b39-798296649839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.745260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafc7c0-0966-4596-8b39-798296649839 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.745549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafc7c0-0966-4596-8b39-798296649839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.745826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafc7c0-0966-4596-8b39-798296649839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.746553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafc7c0-0966-4596-8b39-798296649839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.748580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafc7c0-0966-4596-8b39-798296649839 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.802036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 943/1884] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:08 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:08.803559 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ea213-ba4c-4f33-a571-471c5a7286dc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:08.815696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafc7c0-0966-4596-8b39-798296649839 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:08.824415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ea213-ba4c-4f33-a571-471c5a7286dc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.825360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7eafc7c0-0966-4596-8b39-798296649839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:08.910186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafc7c0-0966-4596-8b39-798296649839 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:08.916905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db758d06-54cc-4f21-ad77-1eadba07d6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.925426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:08.926664 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.927059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.927410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] PATH_INFO: `/v3/projects/84a900bbc61d458ababa1989b99e9eeb` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.929726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a900bbc61d458ababa1989b99e9eeb)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.936955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.943912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db758d06-54cc-4f21-ad77-1eadba07d6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:08.944205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db758d06-54cc-4f21-ad77-1eadba07d6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:08.944493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db758d06-54cc-4f21-ad77-1eadba07d6e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:08.945237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db758d06-54cc-4f21-ad77-1eadba07d6e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:08.959317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db758d06-54cc-4f21-ad77-1eadba07d6e1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:08.992320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 942/1885] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:08 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:08.997921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db758d06-54cc-4f21-ad77-1eadba07d6e1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:09.009380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db758d06-54cc-4f21-ad77-1eadba07d6e1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:09.028416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a900bbc61d458ababa1989b99e9eeb', 'request_id': 'req-e665dba5-fab1-4802-8b2c-2953d59aeb2e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.041538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a900bbc61d458ababa1989b99e9eeb', 'request_id': 'req-e665dba5-fab1-4802-8b2c-2953d59aeb2e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.049400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a900bbc61d458ababa1989b99e9eeb', 'request_id': 'req-e665dba5-fab1-4802-8b2c-2953d59aeb2e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.063407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] The token cache is being invalidate because project 84a900bbc61d458ababa1989b99e9eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:09.064029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.064449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.069654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 944/1886] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:08 2022] DELETE /identity/v3/projects/84a900bbc61d458ababa1989b99e9eeb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:09.070622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:09.072236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:09.156352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665dba5-fab1-4802-8b2c-2953d59aeb2e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:09.161471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:09.162535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.162881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.163163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] PATH_INFO: `/v3/users/ba80cef3b341431ca66139706d148786` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.164906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba80cef3b341431ca66139706d148786)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:09.173472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:09.176098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ec21d-829e-4fd6-b6d3-dbdf687a47f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:09.176840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ec21d-829e-4fd6-b6d3-dbdf687a47f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.177946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ec21d-829e-4fd6-b6d3-dbdf687a47f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.178230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ec21d-829e-4fd6-b6d3-dbdf687a47f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.179018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ec21d-829e-4fd6-b6d3-dbdf687a47f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:09.184040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ec21d-829e-4fd6-b6d3-dbdf687a47f5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:09.184645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:09.211664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 943/1887] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:09.213720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d045c-52b6-4e50-85ec-967d57737ae6 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.214362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d045c-52b6-4e50-85ec-967d57737ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.214902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d045c-52b6-4e50-85ec-967d57737ae6 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.216871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 944/1888] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:42:09.218077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d045c-52b6-4e50-85ec-967d57737ae6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:09.221135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-494d045c-52b6-4e50-85ec-967d57737ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:09.251419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d045c-52b6-4e50-85ec-967d57737ae6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:09.256047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba80cef3b341431ca66139706d148786'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.257219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba80cef3b341431ca66139706d148786'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.262235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5768482-1149-4c5c-8fb0-4e5496d406ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:09.263181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5768482-1149-4c5c-8fb0-4e5496d406ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.263439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5768482-1149-4c5c-8fb0-4e5496d406ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.263766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5768482-1149-4c5c-8fb0-4e5496d406ba admin admin] PATH_INFO: `/v3/projects/6795d04f455e424dac3a848d9b0419cc/users/387c1bed64b046b9a4c89437ea5916bf/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.287149 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5768482-1149-4c5c-8fb0-4e5496d406ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6795d04f455e424dac3a848d9b0419cc, user_id=387c1bed64b046b9a4c89437ea5916bf, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:09.287354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba80cef3b341431ca66139706d148786'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.290924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5768482-1149-4c5c-8fb0-4e5496d406ba admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:09.296263 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba80cef3b341431ca66139706d148786'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.307498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba80cef3b341431ca66139706d148786'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:09.314221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 945/1889] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:09 2022] PUT /identity/v3/projects/6795d04f455e424dac3a848d9b0419cc/users/387c1bed64b046b9a4c89437ea5916bf/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:09.316915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f64f9-5d0e-47b0-bbf1-7ffb5a342c00 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.321308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f64f9-5d0e-47b0-bbf1-7ffb5a342c00 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.322589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 945/1890] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:09 2022] DELETE /identity/v3/users/ba80cef3b341431ca66139706d148786 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:09.323562 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f64f9-5d0e-47b0-bbf1-7ffb5a342c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.323920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:09.338998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:09.363036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-336f64f9-5d0e-47b0-bbf1-7ffb5a342c00 None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:09.364318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-336f64f9-5d0e-47b0-bbf1-7ffb5a342c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:09.438905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:09.474425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 946/1891] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:42:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:09.481999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d188fa-5e93-4125-b6c9-9274d80d9ccc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:09.487414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5286e1-a1c3-4786-8212-945262f735b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:09.488660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5286e1-a1c3-4786-8212-945262f735b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.488953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5286e1-a1c3-4786-8212-945262f735b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.489222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5286e1-a1c3-4786-8212-945262f735b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.490487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5286e1-a1c3-4786-8212-945262f735b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:09.492183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5286e1-a1c3-4786-8212-945262f735b7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:09.510995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 946/1892] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:09.511887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f64f9-5d0e-47b0-bbf1-7ffb5a342c00 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:09.512828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5286e1-a1c3-4786-8212-945262f735b7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:09.515390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1d5286e1-a1c3-4786-8212-945262f735b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:09.529830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f64f9-5d0e-47b0-bbf1-7ffb5a342c00 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:09.542974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5286e1-a1c3-4786-8212-945262f735b7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:09.543911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db42ffc-1eb9-421e-be0c-b05cbd76f076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:09.545521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db42ffc-1eb9-421e-be0c-b05cbd76f076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.545806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db42ffc-1eb9-421e-be0c-b05cbd76f076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.547895 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db42ffc-1eb9-421e-be0c-b05cbd76f076 admin admin] PATH_INFO: `/v3/projects/6795d04f455e424dac3a848d9b0419cc/users/387c1bed64b046b9a4c89437ea5916bf/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.551680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366597b-e8d0-4720-b368-1a7d0e1f9293 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:09.552412 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366597b-e8d0-4720-b368-1a7d0e1f9293 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.552729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366597b-e8d0-4720-b368-1a7d0e1f9293 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.553049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366597b-e8d0-4720-b368-1a7d0e1f9293 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.553323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db42ffc-1eb9-421e-be0c-b05cbd76f076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6795d04f455e424dac3a848d9b0419cc, user_id=387c1bed64b046b9a4c89437ea5916bf, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:09.557065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db42ffc-1eb9-421e-be0c-b05cbd76f076 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:09.574890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366597b-e8d0-4720-b368-1a7d0e1f9293 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:09.576862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366597b-e8d0-4720-b368-1a7d0e1f9293 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:09.579860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 947/1893] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:09 2022] PUT /identity/v3/projects/6795d04f455e424dac3a848d9b0419cc/users/387c1bed64b046b9a4c89437ea5916bf/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:09.602516 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 947/1894] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:09.609859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f75d80-b512-4f0e-8c32-751c83b2d542 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.612018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f75d80-b512-4f0e-8c32-751c83b2d542 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.612314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f75d80-b512-4f0e-8c32-751c83b2d542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.685184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-94f75d80-b512-4f0e-8c32-751c83b2d542 None None] MFA Rules not processed for user `387c1bed64b046b9a4c89437ea5916bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:09.686106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-94f75d80-b512-4f0e-8c32-751c83b2d542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:09.775311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 948/1895] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:09.784973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366597b-e8d0-4720-b368-1a7d0e1f9293 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:09.786705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2366597b-e8d0-4720-b368-1a7d0e1f9293 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:09.889635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366597b-e8d0-4720-b368-1a7d0e1f9293 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:09.913527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:09.988331 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4a518-0022-4d41-9e25-f6340d6f9992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:09.989371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4a518-0022-4d41-9e25-f6340d6f9992 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:09.989863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4a518-0022-4d41-9e25-f6340d6f9992 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:09.991372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4a518-0022-4d41-9e25-f6340d6f9992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:09.994305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-42d4a518-0022-4d41-9e25-f6340d6f9992 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:10.117927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4a518-0022-4d41-9e25-f6340d6f9992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:10.120814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4a518-0022-4d41-9e25-f6340d6f9992 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:10.138655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 948/1896] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:10.542008 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f75d80-b512-4f0e-8c32-751c83b2d542 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:10.544022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-94f75d80-b512-4f0e-8c32-751c83b2d542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:10.712897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f75d80-b512-4f0e-8c32-751c83b2d542 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:10.736723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:10.739026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:10.740273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:10.741250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] PATH_INFO: `/v3/projects/84db565a067c4ec9b76389391f1ecad0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:10.745645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=84db565a067c4ec9b76389391f1ecad0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:10.749186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:10.818551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84db565a067c4ec9b76389391f1ecad0', 'request_id': 'req-9ebe41f0-4164-44d0-ab0e-752325eb71ac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:10.826993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84db565a067c4ec9b76389391f1ecad0', 'request_id': 'req-9ebe41f0-4164-44d0-ab0e-752325eb71ac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:10.834533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:10.837521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84db565a067c4ec9b76389391f1ecad0', 'request_id': 'req-9ebe41f0-4164-44d0-ab0e-752325eb71ac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:10.867376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] The token cache is being invalidate because project 84db565a067c4ec9b76389391f1ecad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:10.867965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:10.869060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ebe41f0-4164-44d0-ab0e-752325eb71ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:10.871299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 949/1897] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:10 2022] DELETE /identity/v3/projects/84db565a067c4ec9b76389391f1ecad0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:10.877474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47858861-2ec1-4ac1-af28-d6635a910943 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:10.877777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47858861-2ec1-4ac1-af28-d6635a910943 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:10.878067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47858861-2ec1-4ac1-af28-d6635a910943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:10.913454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-47858861-2ec1-4ac1-af28-d6635a910943 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:10.914816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-47858861-2ec1-4ac1-af28-d6635a910943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:11.006988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 949/1898] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:11.015677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b4895-51b3-4545-966b-a6717ccb92c9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:11.016028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b4895-51b3-4545-966b-a6717ccb92c9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:11.016309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b4895-51b3-4545-966b-a6717ccb92c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:11.041543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-551b4895-51b3-4545-966b-a6717ccb92c9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:11.042878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-551b4895-51b3-4545-966b-a6717ccb92c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:11.051696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 950/1899] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:11.056129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47858861-2ec1-4ac1-af28-d6635a910943 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:11.070311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47858861-2ec1-4ac1-af28-d6635a910943 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:11.076083 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee2f04-3e0b-473b-b7cc-b1b938ec6fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:11.077225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee2f04-3e0b-473b-b7cc-b1b938ec6fee admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:11.077225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee2f04-3e0b-473b-b7cc-b1b938ec6fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:11.077511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee2f04-3e0b-473b-b7cc-b1b938ec6fee admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:11.078039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee2f04-3e0b-473b-b7cc-b1b938ec6fee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:11.078361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee2f04-3e0b-473b-b7cc-b1b938ec6fee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:11.080167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee2f04-3e0b-473b-b7cc-b1b938ec6fee admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:11.095788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 950/1900] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:42:11.099017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b4895-51b3-4545-966b-a6717ccb92c9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:11.112043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b4895-51b3-4545-966b-a6717ccb92c9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:11.117680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ffd5b-7189-44a2-a059-9dfd52c98c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:11.118384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ffd5b-7189-44a2-a059-9dfd52c98c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:11.118672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ffd5b-7189-44a2-a059-9dfd52c98c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:11.118950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ffd5b-7189-44a2-a059-9dfd52c98c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:11.119737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ffd5b-7189-44a2-a059-9dfd52c98c50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:11.121687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ffd5b-7189-44a2-a059-9dfd52c98c50 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:11.155813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 951/1901] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:11 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:11.160471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee2f04-3e0b-473b-b7cc-b1b938ec6fee admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:11.170737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee2f04-3e0b-473b-b7cc-b1b938ec6fee admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:11.243448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e4ab9-507c-4079-a88e-f849a21c92ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:11.244518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e4ab9-507c-4079-a88e-f849a21c92ba admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:11.245030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e4ab9-507c-4079-a88e-f849a21c92ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:11.245484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e4ab9-507c-4079-a88e-f849a21c92ba admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:11.246588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e4ab9-507c-4079-a88e-f849a21c92ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:11.249150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e4ab9-507c-4079-a88e-f849a21c92ba admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:11.261549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:11.276975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 951/1902] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:11 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:11.281168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ffd5b-7189-44a2-a059-9dfd52c98c50 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:11.291988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ffd5b-7189-44a2-a059-9dfd52c98c50 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:11.296583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf96de-210c-474b-9d6a-fa3dc1a9de43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:11.297257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf96de-210c-474b-9d6a-fa3dc1a9de43 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:11.297579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf96de-210c-474b-9d6a-fa3dc1a9de43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:11.297862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf96de-210c-474b-9d6a-fa3dc1a9de43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:11.298622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf96de-210c-474b-9d6a-fa3dc1a9de43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:11.300283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf96de-210c-474b-9d6a-fa3dc1a9de43 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:11.320068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 952/1903] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:11.321861 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e4ab9-507c-4079-a88e-f849a21c92ba admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:11.339256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e4ab9-507c-4079-a88e-f849a21c92ba admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:11.345096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c12bf-7172-4013-b718-24c0f5ea3c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:11.346011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c12bf-7172-4013-b718-24c0f5ea3c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:11.346393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c12bf-7172-4013-b718-24c0f5ea3c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:11.346798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c12bf-7172-4013-b718-24c0f5ea3c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:11.347635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c12bf-7172-4013-b718-24c0f5ea3c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:11.349714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c12bf-7172-4013-b718-24c0f5ea3c0b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:11.367261 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 952/1904] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:11.368493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf96de-210c-474b-9d6a-fa3dc1a9de43 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:11.381194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf96de-210c-474b-9d6a-fa3dc1a9de43 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:11.388655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458947d8-7e75-4836-9964-88f2f2f7057f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:11.407046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458947d8-7e75-4836-9964-88f2f2f7057f admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:11.407594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458947d8-7e75-4836-9964-88f2f2f7057f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:11.407826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458947d8-7e75-4836-9964-88f2f2f7057f admin admin] PATH_INFO: `/v3/projects/8ff597f1ac8847d9a9941052d6c20aa2/users/333f404b1ea640d4b2b00b23f2861d30/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:11.429487 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458947d8-7e75-4836-9964-88f2f2f7057f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff597f1ac8847d9a9941052d6c20aa2, user_id=333f404b1ea640d4b2b00b23f2861d30, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:11.433100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458947d8-7e75-4836-9964-88f2f2f7057f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:11.459213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 953/1905] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:11 2022] PUT /identity/v3/projects/8ff597f1ac8847d9a9941052d6c20aa2/users/333f404b1ea640d4b2b00b23f2861d30/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:11.471070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b1538-149e-41ab-b975-1f1cde340ec0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:11.471362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b1538-149e-41ab-b975-1f1cde340ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:11.471674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b1538-149e-41ab-b975-1f1cde340ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:11.520282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a90b1538-149e-41ab-b975-1f1cde340ec0 None None] MFA Rules not processed for user `333f404b1ea640d4b2b00b23f2861d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:11.522008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a90b1538-149e-41ab-b975-1f1cde340ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:11.693301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 953/1906] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:11.698303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458947d8-7e75-4836-9964-88f2f2f7057f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:11.881863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458947d8-7e75-4836-9964-88f2f2f7057f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:11.892446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fae00-0348-4830-a271-0b0994b57911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:11.893208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fae00-0348-4830-a271-0b0994b57911 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:11.893490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fae00-0348-4830-a271-0b0994b57911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:11.893935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fae00-0348-4830-a271-0b0994b57911 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:11.897069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fae00-0348-4830-a271-0b0994b57911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:11.899218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fae00-0348-4830-a271-0b0994b57911 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:11.950530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 954/1907] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:11 2022] POST /identity/v3/projects => generated 385 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:11.951789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b1538-149e-41ab-b975-1f1cde340ec0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:11.964530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b1538-149e-41ab-b975-1f1cde340ec0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:12.084723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f055-e019-440d-b571-1f78919a40ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:12.085574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f055-e019-440d-b571-1f78919a40ef admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:12.085935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f055-e019-440d-b571-1f78919a40ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:12.086288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f055-e019-440d-b571-1f78919a40ef admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:12.087305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872f055-e019-440d-b571-1f78919a40ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:12.090242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872f055-e019-440d-b571-1f78919a40ef admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:12.122944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 954/1908] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:11 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:12.127014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fae00-0348-4830-a271-0b0994b57911 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:12.145303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fae00-0348-4830-a271-0b0994b57911 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:12.158459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef20c8-f642-4a4e-acca-ca0b020d6d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:12.159271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef20c8-f642-4a4e-acca-ca0b020d6d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:12.159684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef20c8-f642-4a4e-acca-ca0b020d6d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:12.160077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef20c8-f642-4a4e-acca-ca0b020d6d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:12.160934 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef20c8-f642-4a4e-acca-ca0b020d6d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:12.162619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef20c8-f642-4a4e-acca-ca0b020d6d66 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:12.182122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f055-e019-440d-b571-1f78919a40ef admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:12.192246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 955/1909] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:12.198545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f055-e019-440d-b571-1f78919a40ef admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:12.204403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6f76d-58a0-4719-880a-9d96685fba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:12.205321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6f76d-58a0-4719-880a-9d96685fba9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:12.205628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6f76d-58a0-4719-880a-9d96685fba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:12.205961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6f76d-58a0-4719-880a-9d96685fba9e admin admin] PATH_INFO: `/v3/projects/5b6a86a2811c4aeea394f45216aee71a/users/080f690cc4f14734bfbf8639b1186aea/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:12.262160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b6f76d-58a0-4719-880a-9d96685fba9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6a86a2811c4aeea394f45216aee71a, user_id=080f690cc4f14734bfbf8639b1186aea, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:12.267025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b6f76d-58a0-4719-880a-9d96685fba9e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:12.305656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef20c8-f642-4a4e-acca-ca0b020d6d66 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:12.306095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 955/1910] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:12 2022] PUT /identity/v3/projects/5b6a86a2811c4aeea394f45216aee71a/users/080f690cc4f14734bfbf8639b1186aea/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:12.316576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef20c8-f642-4a4e-acca-ca0b020d6d66 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:12.440395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:12.457245 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7953ab-d8b1-4b1f-b251-2f44358d62f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:12.457871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7953ab-d8b1-4b1f-b251-2f44358d62f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:12.460703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7953ab-d8b1-4b1f-b251-2f44358d62f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:12.461039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7953ab-d8b1-4b1f-b251-2f44358d62f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:12.465009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7953ab-d8b1-4b1f-b251-2f44358d62f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:12.466932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7953ab-d8b1-4b1f-b251-2f44358d62f1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:12.505346 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6f76d-58a0-4719-880a-9d96685fba9e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:12.516215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 956/1911] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:12.521417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6f76d-58a0-4719-880a-9d96685fba9e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:12.531037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994be6ae-001c-4f6e-bfdb-b782cf558dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:12.534152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994be6ae-001c-4f6e-bfdb-b782cf558dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:12.534620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994be6ae-001c-4f6e-bfdb-b782cf558dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:12.535025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994be6ae-001c-4f6e-bfdb-b782cf558dde admin admin] PATH_INFO: `/v3/projects/5b6a86a2811c4aeea394f45216aee71a/users/080f690cc4f14734bfbf8639b1186aea/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:12.548485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994be6ae-001c-4f6e-bfdb-b782cf558dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6a86a2811c4aeea394f45216aee71a, user_id=080f690cc4f14734bfbf8639b1186aea, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:12.551331 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994be6ae-001c-4f6e-bfdb-b782cf558dde admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:12.575892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 956/1912] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:12 2022] PUT /identity/v3/projects/5b6a86a2811c4aeea394f45216aee71a/users/080f690cc4f14734bfbf8639b1186aea/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:12.585239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc938b8-c2d3-40ad-afda-10aad096a393 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:12.586115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc938b8-c2d3-40ad-afda-10aad096a393 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:12.586625 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc938b8-c2d3-40ad-afda-10aad096a393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:12.695863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2cc938b8-c2d3-40ad-afda-10aad096a393 None None] MFA Rules not processed for user `080f690cc4f14734bfbf8639b1186aea`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:12.697098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2cc938b8-c2d3-40ad-afda-10aad096a393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:12.861742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 957/1913] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:12.884395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994be6ae-001c-4f6e-bfdb-b782cf558dde admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:12.887393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-994be6ae-001c-4f6e-bfdb-b782cf558dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:12.948609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:13.041812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994be6ae-001c-4f6e-bfdb-b782cf558dde admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:13.055349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15acc4-3089-4d4c-978c-83c67071642f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:13.056938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15acc4-3089-4d4c-978c-83c67071642f service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:13.057230 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15acc4-3089-4d4c-978c-83c67071642f service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:13.057514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15acc4-3089-4d4c-978c-83c67071642f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:13.086989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15acc4-3089-4d4c-978c-83c67071642f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:13.089759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15acc4-3089-4d4c-978c-83c67071642f service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:13.119958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 957/1914] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:13.231799 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc938b8-c2d3-40ad-afda-10aad096a393 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:13.322070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc938b8-c2d3-40ad-afda-10aad096a393 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:13.328923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:13.329993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:13.330370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:13.330844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] PATH_INFO: `/v3/users/333f404b1ea640d4b2b00b23f2861d30` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:13.332165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333f404b1ea640d4b2b00b23f2861d30)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:13.334427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:13.396160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333f404b1ea640d4b2b00b23f2861d30'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:13.396836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333f404b1ea640d4b2b00b23f2861d30'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:13.411713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333f404b1ea640d4b2b00b23f2861d30'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:13.423061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333f404b1ea640d4b2b00b23f2861d30'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:13.423944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333f404b1ea640d4b2b00b23f2861d30'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:13.440482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:13.455271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 958/1915] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:13 2022] DELETE /identity/v3/users/333f404b1ea640d4b2b00b23f2861d30 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:13.459949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4542-c62e-4ea2-ab4f-9675aa48f4ca None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:13.460431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4542-c62e-4ea2-ab4f-9675aa48f4ca None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:13.460705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4542-c62e-4ea2-ab4f-9675aa48f4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:13.478870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-186e4542-c62e-4ea2-ab4f-9675aa48f4ca None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:13.480164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-186e4542-c62e-4ea2-ab4f-9675aa48f4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:13.571499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 958/1916] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:13.583277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:13.585906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:13.679352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e801b42-3b28-465e-87eb-76a7bdd54e67 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:13.691047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f413c-ec64-447d-b31a-b980be4f3142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:13.691945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f413c-ec64-447d-b31a-b980be4f3142 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:13.692813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f413c-ec64-447d-b31a-b980be4f3142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:13.694354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f413c-ec64-447d-b31a-b980be4f3142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:13.707591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f413c-ec64-447d-b31a-b980be4f3142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:13.709067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f413c-ec64-447d-b31a-b980be4f3142 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:13.727875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 959/1917] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:14.212343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4542-c62e-4ea2-ab4f-9675aa48f4ca None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:14.213991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-186e4542-c62e-4ea2-ab4f-9675aa48f4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:14.235278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4542-c62e-4ea2-ab4f-9675aa48f4ca None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:14.249115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:14.250231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:14.250676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:14.250964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] PATH_INFO: `/v3/projects/8ff597f1ac8847d9a9941052d6c20aa2` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:14.252204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff597f1ac8847d9a9941052d6c20aa2)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:14.254220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:14.303819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff597f1ac8847d9a9941052d6c20aa2', 'request_id': 'req-855dcfd5-7105-4462-8538-3af5dfc27277'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.325810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff597f1ac8847d9a9941052d6c20aa2', 'request_id': 'req-855dcfd5-7105-4462-8538-3af5dfc27277'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.334748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff597f1ac8847d9a9941052d6c20aa2', 'request_id': 'req-855dcfd5-7105-4462-8538-3af5dfc27277'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.349853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] The token cache is being invalidate because project 8ff597f1ac8847d9a9941052d6c20aa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:14.350373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.351510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.357411 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 959/1918] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:14 2022] DELETE /identity/v3/projects/8ff597f1ac8847d9a9941052d6c20aa2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:14.358422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f413c-ec64-447d-b31a-b980be4f3142 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:14.371633 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f413c-ec64-447d-b31a-b980be4f3142 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:14.497778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:14.499019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:14.499535 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:14.499958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] PATH_INFO: `/v3/users/080f690cc4f14734bfbf8639b1186aea` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:14.501467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=080f690cc4f14734bfbf8639b1186aea)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:14.503815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:14.545880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:14.547955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:14.565481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855dcfd5-7105-4462-8538-3af5dfc27277 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:14.578158 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:14.579341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:14.579652 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:14.579940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] PATH_INFO: `/v3/users/f12d7ce7904e41b79f3b3e9ecb086700` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:14.583339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f12d7ce7904e41b79f3b3e9ecb086700)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:14.587044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:14.592664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080f690cc4f14734bfbf8639b1186aea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.594461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080f690cc4f14734bfbf8639b1186aea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.616100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080f690cc4f14734bfbf8639b1186aea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.626465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080f690cc4f14734bfbf8639b1186aea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.627552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b3ce14a-8566-43a4-b036-8ce8594f6b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080f690cc4f14734bfbf8639b1186aea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.650739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 960/1919] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:14 2022] DELETE /identity/v3/users/080f690cc4f14734bfbf8639b1186aea => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:14.663558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:14.667521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f12d7ce7904e41b79f3b3e9ecb086700'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.668679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f12d7ce7904e41b79f3b3e9ecb086700'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.684926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f12d7ce7904e41b79f3b3e9ecb086700'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.693958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f12d7ce7904e41b79f3b3e9ecb086700'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.704072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f12d7ce7904e41b79f3b3e9ecb086700'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:14.710889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 960/1920] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:14 2022] DELETE /identity/v3/users/f12d7ce7904e41b79f3b3e9ecb086700 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:14.716061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c8a8f-81cd-415f-a737-4926938ab051 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:14.716462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c8a8f-81cd-415f-a737-4926938ab051 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:14.717429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c8a8f-81cd-415f-a737-4926938ab051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:14.736100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1a4c8a8f-81cd-415f-a737-4926938ab051 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:14.737636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1a4c8a8f-81cd-415f-a737-4926938ab051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:14.774402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:14.869736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 961/1921] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:14.893132 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:14.895052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:14.978590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ac50d-ad2b-4f95-881f-a45441903602 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:14.983286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081f3e3-0bf8-48a2-8262-c18f9f9c5fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:14.984001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081f3e3-0bf8-48a2-8262-c18f9f9c5fad service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:14.984284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081f3e3-0bf8-48a2-8262-c18f9f9c5fad service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:14.984567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081f3e3-0bf8-48a2-8262-c18f9f9c5fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:14.997314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081f3e3-0bf8-48a2-8262-c18f9f9c5fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:14.998784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081f3e3-0bf8-48a2-8262-c18f9f9c5fad service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:15.010046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 961/1922] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:15.344422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c8a8f-81cd-415f-a737-4926938ab051 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:15.345848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1a4c8a8f-81cd-415f-a737-4926938ab051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:15.375696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081f3e3-0bf8-48a2-8262-c18f9f9c5fad service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:15.377515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7081f3e3-0bf8-48a2-8262-c18f9f9c5fad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:15.400852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081f3e3-0bf8-48a2-8262-c18f9f9c5fad service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:15.408722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:15.410250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:15.410826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:15.410975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] PATH_INFO: `/v3/projects/510401c9ecba4cf9990a3cc7d6d963ad` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:15.418989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510401c9ecba4cf9990a3cc7d6d963ad)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:15.421415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:15.444726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c8a8f-81cd-415f-a737-4926938ab051 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:15.450508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca82579-0935-4d68-aa12-8f25ab58e478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:15.451536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca82579-0935-4d68-aa12-8f25ab58e478 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:15.452026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca82579-0935-4d68-aa12-8f25ab58e478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:15.452500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca82579-0935-4d68-aa12-8f25ab58e478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:15.454763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0ca82579-0935-4d68-aa12-8f25ab58e478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:15.490714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510401c9ecba4cf9990a3cc7d6d963ad', 'request_id': 'req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.502743 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510401c9ecba4cf9990a3cc7d6d963ad', 'request_id': 'req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.509218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510401c9ecba4cf9990a3cc7d6d963ad', 'request_id': 'req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.523653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] The token cache is being invalidate because project 510401c9ecba4cf9990a3cc7d6d963ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:15.524022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.524786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.529130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 962/1923] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:15 2022] DELETE /identity/v3/projects/510401c9ecba4cf9990a3cc7d6d963ad => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:15.530229 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:15.532166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:15.572886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca82579-0935-4d68-aa12-8f25ab58e478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:15.574757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca82579-0935-4d68-aa12-8f25ab58e478 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:15.585817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 962/1924] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:15.586992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca82579-0935-4d68-aa12-8f25ab58e478 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:15.597255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:15.597711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca82579-0935-4d68-aa12-8f25ab58e478 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:15.622529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e0ca4-9d19-4a51-bf50-e73b62e85af6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:15.629364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:15.630336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:15.630690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:15.631200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] PATH_INFO: `/v3/projects/5b6a86a2811c4aeea394f45216aee71a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:15.632375 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b6a86a2811c4aeea394f45216aee71a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:15.634381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:15.676923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:15.677859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:15.678150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:15.678486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] PATH_INFO: `/v3/users/1529aafd9a184bfab3a444ecfdf1cfc9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:15.679763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1529aafd9a184bfab3a444ecfdf1cfc9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:15.681750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:15.696660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:15.710634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b6a86a2811c4aeea394f45216aee71a', 'request_id': 'req-9dd7d71e-0837-414c-b179-232bfc25d03a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.721728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b6a86a2811c4aeea394f45216aee71a', 'request_id': 'req-9dd7d71e-0837-414c-b179-232bfc25d03a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.728432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b6a86a2811c4aeea394f45216aee71a', 'request_id': 'req-9dd7d71e-0837-414c-b179-232bfc25d03a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.746139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] The token cache is being invalidate because project 5b6a86a2811c4aeea394f45216aee71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:15.746588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.747875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9dd7d71e-0837-414c-b179-232bfc25d03a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.748051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1529aafd9a184bfab3a444ecfdf1cfc9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.748669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1529aafd9a184bfab3a444ecfdf1cfc9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.749867 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 963/1925] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:15 2022] DELETE /identity/v3/projects/5b6a86a2811c4aeea394f45216aee71a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:15.755609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b0bd0-881e-4ea5-bf1a-8e3771f084ae None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:15.755967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b0bd0-881e-4ea5-bf1a-8e3771f084ae None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:15.756316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b0bd0-881e-4ea5-bf1a-8e3771f084ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:15.763152 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1529aafd9a184bfab3a444ecfdf1cfc9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.775332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1529aafd9a184bfab3a444ecfdf1cfc9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.776354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fe4932d3-9ea6-4dae-a608-f6c024351ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1529aafd9a184bfab3a444ecfdf1cfc9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:15.790874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9d4b0bd0-881e-4ea5-bf1a-8e3771f084ae None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:15.792791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9d4b0bd0-881e-4ea5-bf1a-8e3771f084ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:15.809477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 963/1926] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:15 2022] DELETE /identity/v3/users/1529aafd9a184bfab3a444ecfdf1cfc9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:15.955740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 964/1927] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:15.958280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69c598-372c-4fa3-9fa3-24b7ef620daf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:15.960676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69c598-372c-4fa3-9fa3-24b7ef620daf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:15.963405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69c598-372c-4fa3-9fa3-24b7ef620daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:15.994378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3e69c598-372c-4fa3-9fa3-24b7ef620daf None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:15.995634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3e69c598-372c-4fa3-9fa3-24b7ef620daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:16.079404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 964/1928] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:16.083483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b0bd0-881e-4ea5-bf1a-8e3771f084ae None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.099652 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b0bd0-881e-4ea5-bf1a-8e3771f084ae None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.104434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20958215-4e85-4599-864f-c4ee2b1dd275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.105215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958215-4e85-4599-864f-c4ee2b1dd275 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.105977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958215-4e85-4599-864f-c4ee2b1dd275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.105977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958215-4e85-4599-864f-c4ee2b1dd275 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.106777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20958215-4e85-4599-864f-c4ee2b1dd275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:16.107211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20958215-4e85-4599-864f-c4ee2b1dd275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.109386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20958215-4e85-4599-864f-c4ee2b1dd275 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.122756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 965/1929] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:42:16.132994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69c598-372c-4fa3-9fa3-24b7ef620daf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.145047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69c598-372c-4fa3-9fa3-24b7ef620daf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.151357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a87696-5383-4ddc-a282-c6c2f53ca131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.152555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a87696-5383-4ddc-a282-c6c2f53ca131 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.152937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a87696-5383-4ddc-a282-c6c2f53ca131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.153308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a87696-5383-4ddc-a282-c6c2f53ca131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.154085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a87696-5383-4ddc-a282-c6c2f53ca131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.156061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a87696-5383-4ddc-a282-c6c2f53ca131 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.186592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 965/1930] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:16 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:16.190649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20958215-4e85-4599-864f-c4ee2b1dd275 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.202311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20958215-4e85-4599-864f-c4ee2b1dd275 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.284395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a87696-5383-4ddc-a282-c6c2f53ca131 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.289204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-86a87696-5383-4ddc-a282-c6c2f53ca131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:16.309618 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb9abb-e09f-4be0-b99b-c86b2d7d9318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.310395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb9abb-e09f-4be0-b99b-c86b2d7d9318 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.310763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb9abb-e09f-4be0-b99b-c86b2d7d9318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.311111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb9abb-e09f-4be0-b99b-c86b2d7d9318 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.312216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb9abb-e09f-4be0-b99b-c86b2d7d9318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.314528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb9abb-e09f-4be0-b99b-c86b2d7d9318 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.326590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:16.343787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 966/1931] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:16 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:16.346413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb9abb-e09f-4be0-b99b-c86b2d7d9318 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.357233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb9abb-e09f-4be0-b99b-c86b2d7d9318 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.362103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd61a4c-2d3f-4485-bf68-2c9ab3eadb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.362874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd61a4c-2d3f-4485-bf68-2c9ab3eadb83 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.363231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd61a4c-2d3f-4485-bf68-2c9ab3eadb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.363647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd61a4c-2d3f-4485-bf68-2c9ab3eadb83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.364489 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd61a4c-2d3f-4485-bf68-2c9ab3eadb83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.366196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd61a4c-2d3f-4485-bf68-2c9ab3eadb83 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.381163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 967/1932] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:16.382765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd61a4c-2d3f-4485-bf68-2c9ab3eadb83 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.393329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd61a4c-2d3f-4485-bf68-2c9ab3eadb83 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.399378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9b045-17ec-45df-af39-6bf1a18bd6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.400191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9b045-17ec-45df-af39-6bf1a18bd6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.400575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9b045-17ec-45df-af39-6bf1a18bd6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.400922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9b045-17ec-45df-af39-6bf1a18bd6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.402290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9b045-17ec-45df-af39-6bf1a18bd6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.403424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9b045-17ec-45df-af39-6bf1a18bd6f9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.406933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a87696-5383-4ddc-a282-c6c2f53ca131 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.414270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.417941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 968/1933] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:16.418185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.418185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.418185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] PATH_INFO: `/v3/projects/9b0d59c49c384d8f90e0f28f7ec406c6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.418185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b0d59c49c384d8f90e0f28f7ec406c6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.419964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.420735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9b045-17ec-45df-af39-6bf1a18bd6f9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.431447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:16.434748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9b045-17ec-45df-af39-6bf1a18bd6f9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.441795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259e2b8-c33d-4e61-a3c2-516847a61245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.444427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259e2b8-c33d-4e61-a3c2-516847a61245 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.445134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259e2b8-c33d-4e61-a3c2-516847a61245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.445672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259e2b8-c33d-4e61-a3c2-516847a61245 admin admin] PATH_INFO: `/v3/projects/25e501dce17d46b4bb53c0a6782da171/users/755a639319e74b5483a6333ca458290e/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.476294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259e2b8-c33d-4e61-a3c2-516847a61245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e501dce17d46b4bb53c0a6782da171, user_id=755a639319e74b5483a6333ca458290e, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.479299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259e2b8-c33d-4e61-a3c2-516847a61245 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.484905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b0d59c49c384d8f90e0f28f7ec406c6', 'request_id': 'req-dfc46b84-e24e-48a0-9514-2e5d355bf080'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:16.493367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b0d59c49c384d8f90e0f28f7ec406c6', 'request_id': 'req-dfc46b84-e24e-48a0-9514-2e5d355bf080'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:16.499593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b0d59c49c384d8f90e0f28f7ec406c6', 'request_id': 'req-dfc46b84-e24e-48a0-9514-2e5d355bf080'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:16.509163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 969/1934] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:16 2022] PUT /identity/v3/projects/25e501dce17d46b4bb53c0a6782da171/users/755a639319e74b5483a6333ca458290e/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:16.512453 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] The token cache is being invalidate because project 9b0d59c49c384d8f90e0f28f7ec406c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:16.512744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:16.513386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfc46b84-e24e-48a0-9514-2e5d355bf080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:16.515161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 966/1935] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:16 2022] DELETE /identity/v3/projects/9b0d59c49c384d8f90e0f28f7ec406c6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:16.519809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da57b72-9ba4-48b5-aec4-ab780a359c22 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.519931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f486b98-a8c8-400e-9d15-e4dcd5262f61 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.520105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da57b72-9ba4-48b5-aec4-ab780a359c22 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.520195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f486b98-a8c8-400e-9d15-e4dcd5262f61 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.520389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da57b72-9ba4-48b5-aec4-ab780a359c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.520496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f486b98-a8c8-400e-9d15-e4dcd5262f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.538929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5da57b72-9ba4-48b5-aec4-ab780a359c22 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:16.540596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5da57b72-9ba4-48b5-aec4-ab780a359c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:16.547792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0f486b98-a8c8-400e-9d15-e4dcd5262f61 None None] MFA Rules not processed for user `755a639319e74b5483a6333ca458290e`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:16.549054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0f486b98-a8c8-400e-9d15-e4dcd5262f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:16.628208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 967/1936] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:16.629217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f486b98-a8c8-400e-9d15-e4dcd5262f61 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.630541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0f486b98-a8c8-400e-9d15-e4dcd5262f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:16.635032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 970/1937] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:16.642658 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59369c39-8a1d-457c-a142-9f9eec3adcf7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.642967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59369c39-8a1d-457c-a142-9f9eec3adcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.643929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59369c39-8a1d-457c-a142-9f9eec3adcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.648042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f486b98-a8c8-400e-9d15-e4dcd5262f61 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.653538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d42d-f236-44f2-8486-90aab6fa5eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.654288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d42d-f236-44f2-8486-90aab6fa5eac admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.654668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d42d-f236-44f2-8486-90aab6fa5eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.655555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d42d-f236-44f2-8486-90aab6fa5eac admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.656151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d42d-f236-44f2-8486-90aab6fa5eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.658261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d42d-f236-44f2-8486-90aab6fa5eac admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.661353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-59369c39-8a1d-457c-a142-9f9eec3adcf7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:16.662644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-59369c39-8a1d-457c-a142-9f9eec3adcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:16.670639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 971/1938] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:16.675080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59369c39-8a1d-457c-a142-9f9eec3adcf7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.685010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59369c39-8a1d-457c-a142-9f9eec3adcf7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.689727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639b18c-50ba-40f6-b6f0-407df0b95948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.690395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b18c-50ba-40f6-b6f0-407df0b95948 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.690676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b18c-50ba-40f6-b6f0-407df0b95948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.690949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b18c-50ba-40f6-b6f0-407df0b95948 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.691718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639b18c-50ba-40f6-b6f0-407df0b95948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:16.692114 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639b18c-50ba-40f6-b6f0-407df0b95948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.693436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 968/1939] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:16 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:16.694238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639b18c-50ba-40f6-b6f0-407df0b95948 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.698323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d42d-f236-44f2-8486-90aab6fa5eac admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.707684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 972/1940] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:42:16.711958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639b18c-50ba-40f6-b6f0-407df0b95948 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.723432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639b18c-50ba-40f6-b6f0-407df0b95948 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.771176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d42d-f236-44f2-8486-90aab6fa5eac admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.777642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0740d4a-0e63-41dc-a495-ab6032afe977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.778576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0740d4a-0e63-41dc-a495-ab6032afe977 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.778858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0740d4a-0e63-41dc-a495-ab6032afe977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.779134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0740d4a-0e63-41dc-a495-ab6032afe977 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.780058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0740d4a-0e63-41dc-a495-ab6032afe977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.782788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0740d4a-0e63-41dc-a495-ab6032afe977 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.790641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d9354-f547-4801-b2c3-8cd9ae4d42fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.791378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d9354-f547-4801-b2c3-8cd9ae4d42fb admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.791709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d9354-f547-4801-b2c3-8cd9ae4d42fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.791995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d9354-f547-4801-b2c3-8cd9ae4d42fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.792911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d9354-f547-4801-b2c3-8cd9ae4d42fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.795197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:16.795309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d9354-f547-4801-b2c3-8cd9ae4d42fb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.811263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 969/1941] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:16 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:16.812305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0740d4a-0e63-41dc-a495-ab6032afe977 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.823420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0740d4a-0e63-41dc-a495-ab6032afe977 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.827260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 973/1942] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:16 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:16.831661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe7634-42be-4dc1-8d61-36574cf01618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.832351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe7634-42be-4dc1-8d61-36574cf01618 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.832632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe7634-42be-4dc1-8d61-36574cf01618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.832938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe7634-42be-4dc1-8d61-36574cf01618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.833671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe7634-42be-4dc1-8d61-36574cf01618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.836144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe7634-42be-4dc1-8d61-36574cf01618 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.840073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d9354-f547-4801-b2c3-8cd9ae4d42fb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.852496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 970/1943] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:16.853785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe7634-42be-4dc1-8d61-36574cf01618 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.864517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe7634-42be-4dc1-8d61-36574cf01618 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.866842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:16.920042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d9354-f547-4801-b2c3-8cd9ae4d42fb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:16.924839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee618a54-9e94-40a0-9888-6b915eed821f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.925528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee618a54-9e94-40a0-9888-6b915eed821f admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.925820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee618a54-9e94-40a0-9888-6b915eed821f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.926105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee618a54-9e94-40a0-9888-6b915eed821f admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.926807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee618a54-9e94-40a0-9888-6b915eed821f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.928826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee618a54-9e94-40a0-9888-6b915eed821f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.929924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ee4fb-3a41-4b22-aa15-e6fe3f65afda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:16.930807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ee4fb-3a41-4b22-aa15-e6fe3f65afda admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:16.931405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ee4fb-3a41-4b22-aa15-e6fe3f65afda admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:16.931405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ee4fb-3a41-4b22-aa15-e6fe3f65afda admin admin] PATH_INFO: `/v3/projects/651f065924734074a73e3d2e7093f303/users/d714affe44e344c293e85647a9134b01/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:16.951034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ee4fb-3a41-4b22-aa15-e6fe3f65afda admin admin] RBAC: Authorizing `identity:create_grant(project_id=651f065924734074a73e3d2e7093f303, user_id=d714affe44e344c293e85647a9134b01, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:16.954161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ee4fb-3a41-4b22-aa15-e6fe3f65afda admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:16.962792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 974/1944] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:16 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:16.966831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee618a54-9e94-40a0-9888-6b915eed821f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.968827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 971/1945] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:16 2022] PUT /identity/v3/projects/651f065924734074a73e3d2e7093f303/users/d714affe44e344c293e85647a9134b01/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:16.972807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ee4fb-3a41-4b22-aa15-e6fe3f65afda admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:16.976781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee618a54-9e94-40a0-9888-6b915eed821f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.032032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ee4fb-3a41-4b22-aa15-e6fe3f65afda admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.032926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1768bc-3e08-4dc1-a2f6-cbee9c4bdc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.033608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1768bc-3e08-4dc1-a2f6-cbee9c4bdc16 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.033899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1768bc-3e08-4dc1-a2f6-cbee9c4bdc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.034183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1768bc-3e08-4dc1-a2f6-cbee9c4bdc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.034942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1768bc-3e08-4dc1-a2f6-cbee9c4bdc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.036629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1768bc-3e08-4dc1-a2f6-cbee9c4bdc16 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.036956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15ca8e-4984-4a72-92e6-72fdc6923fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.037661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15ca8e-4984-4a72-92e6-72fdc6923fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.037947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15ca8e-4984-4a72-92e6-72fdc6923fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.038259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15ca8e-4984-4a72-92e6-72fdc6923fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.039019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15ca8e-4984-4a72-92e6-72fdc6923fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.040776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15ca8e-4984-4a72-92e6-72fdc6923fdd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.047061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 975/1946] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:17.053655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 972/1947] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:17.053830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1768bc-3e08-4dc1-a2f6-cbee9c4bdc16 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.056368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15ca8e-4984-4a72-92e6-72fdc6923fdd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.066926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1768bc-3e08-4dc1-a2f6-cbee9c4bdc16 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.069972 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15ca8e-4984-4a72-92e6-72fdc6923fdd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.071961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5da6a1-1188-4e54-848b-cff44b035086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.072638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5da6a1-1188-4e54-848b-cff44b035086 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.072928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5da6a1-1188-4e54-848b-cff44b035086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.073210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5da6a1-1188-4e54-848b-cff44b035086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.073942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5da6a1-1188-4e54-848b-cff44b035086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.074547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ea857-d7a6-49c4-aefc-ce8164a670a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.075484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ea857-d7a6-49c4-aefc-ce8164a670a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.075625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5da6a1-1188-4e54-848b-cff44b035086 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.075782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ea857-d7a6-49c4-aefc-ce8164a670a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.076088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ea857-d7a6-49c4-aefc-ce8164a670a9 admin admin] PATH_INFO: `/v3/projects/651f065924734074a73e3d2e7093f303/users/d714affe44e344c293e85647a9134b01/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.078036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ea857-d7a6-49c4-aefc-ce8164a670a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=651f065924734074a73e3d2e7093f303, user_id=d714affe44e344c293e85647a9134b01, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.080762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ea857-d7a6-49c4-aefc-ce8164a670a9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.089780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 976/1948] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:17.090695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5da6a1-1188-4e54-848b-cff44b035086 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.101536 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 973/1949] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:17 2022] PUT /identity/v3/projects/651f065924734074a73e3d2e7093f303/users/d714affe44e344c293e85647a9134b01/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:17.102999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be7e5-92a8-4c4b-8a74-eb7830b547c5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.103303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be7e5-92a8-4c4b-8a74-eb7830b547c5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.103621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be7e5-92a8-4c4b-8a74-eb7830b547c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.106657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5da6a1-1188-4e54-848b-cff44b035086 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.144065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-595be7e5-92a8-4c4b-8a74-eb7830b547c5 None None] MFA Rules not processed for user `d714affe44e344c293e85647a9134b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:17.145185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-595be7e5-92a8-4c4b-8a74-eb7830b547c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:17.173867 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca507077-714f-4b25-a17c-f4ebac472838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.174736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca507077-714f-4b25-a17c-f4ebac472838 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.175025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca507077-714f-4b25-a17c-f4ebac472838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.175361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca507077-714f-4b25-a17c-f4ebac472838 admin admin] PATH_INFO: `/v3/projects/898873b33a414016bb843230d1a18486/users/07b8ae99f8b14168aaa490d299272389/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.193278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca507077-714f-4b25-a17c-f4ebac472838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=898873b33a414016bb843230d1a18486, user_id=07b8ae99f8b14168aaa490d299272389, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.196008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca507077-714f-4b25-a17c-f4ebac472838 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.198186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:17.226708 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 977/1950] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:17 2022] PUT /identity/v3/projects/898873b33a414016bb843230d1a18486/users/07b8ae99f8b14168aaa490d299272389/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:17.226708 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfe4db-3b70-4144-a207-8f16356b8e49 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.226708 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfe4db-3b70-4144-a207-8f16356b8e49 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.226708 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfe4db-3b70-4144-a207-8f16356b8e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.230134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 974/1951] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:17.243567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595be7e5-92a8-4c4b-8a74-eb7830b547c5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.245461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-595be7e5-92a8-4c4b-8a74-eb7830b547c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:17.250227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:17.253894 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-49bfe4db-3b70-4144-a207-8f16356b8e49 None None] MFA Rules not processed for user `07b8ae99f8b14168aaa490d299272389`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:17.256916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-49bfe4db-3b70-4144-a207-8f16356b8e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:17.314601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595be7e5-92a8-4c4b-8a74-eb7830b547c5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.347617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 978/1952] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:17.352714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfe4db-3b70-4144-a207-8f16356b8e49 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.384857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b008d1d-1108-445c-a5cd-1bf0450fc1d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.385751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b008d1d-1108-445c-a5cd-1bf0450fc1d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.386119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b008d1d-1108-445c-a5cd-1bf0450fc1d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.386467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b008d1d-1108-445c-a5cd-1bf0450fc1d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.427164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfe4db-3b70-4144-a207-8f16356b8e49 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.431722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3edd0-33d0-43c4-858b-08fe69ab2928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.432399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3edd0-33d0-43c4-858b-08fe69ab2928 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.432686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3edd0-33d0-43c4-858b-08fe69ab2928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.432969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3edd0-33d0-43c4-858b-08fe69ab2928 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.433661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3edd0-33d0-43c4-858b-08fe69ab2928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.435637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3edd0-33d0-43c4-858b-08fe69ab2928 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.460928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b008d1d-1108-445c-a5cd-1bf0450fc1d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.464682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 979/1953] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:17 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:17.464807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b008d1d-1108-445c-a5cd-1bf0450fc1d6 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.465699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3edd0-33d0-43c4-858b-08fe69ab2928 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.472847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 975/1954] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:17.534909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3edd0-33d0-43c4-858b-08fe69ab2928 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.535957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b008d1d-1108-445c-a5cd-1bf0450fc1d6 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.538280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3b008d1d-1108-445c-a5cd-1bf0450fc1d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:17.541890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636742a0-0b8f-4277-93ab-86fdaecbbf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.542623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636742a0-0b8f-4277-93ab-86fdaecbbf60 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.542853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636742a0-0b8f-4277-93ab-86fdaecbbf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.543137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636742a0-0b8f-4277-93ab-86fdaecbbf60 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.543876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636742a0-0b8f-4277-93ab-86fdaecbbf60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.545835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636742a0-0b8f-4277-93ab-86fdaecbbf60 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.553487 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b008d1d-1108-445c-a5cd-1bf0450fc1d6 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.601310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 980/1955] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:17 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:17.603525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636742a0-0b8f-4277-93ab-86fdaecbbf60 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.623170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636742a0-0b8f-4277-93ab-86fdaecbbf60 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.629989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b012e0-5bd2-4b42-bd43-dcf50e51d26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.630748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b012e0-5bd2-4b42-bd43-dcf50e51d26d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.631042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b012e0-5bd2-4b42-bd43-dcf50e51d26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.631386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b012e0-5bd2-4b42-bd43-dcf50e51d26d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.632608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b012e0-5bd2-4b42-bd43-dcf50e51d26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.634350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b012e0-5bd2-4b42-bd43-dcf50e51d26d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.634878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:17.653499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 981/1956] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:17.657860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b012e0-5bd2-4b42-bd43-dcf50e51d26d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.660673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce784f74-0ea1-4ccd-94f6-7871949b4945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.661356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784f74-0ea1-4ccd-94f6-7871949b4945 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.661697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784f74-0ea1-4ccd-94f6-7871949b4945 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.661983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784f74-0ea1-4ccd-94f6-7871949b4945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.674190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b012e0-5bd2-4b42-bd43-dcf50e51d26d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.680274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523658a5-9835-43d7-a106-0728b06ee53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.680956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523658a5-9835-43d7-a106-0728b06ee53e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.681246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523658a5-9835-43d7-a106-0728b06ee53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.681525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523658a5-9835-43d7-a106-0728b06ee53e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.682286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523658a5-9835-43d7-a106-0728b06ee53e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.684107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523658a5-9835-43d7-a106-0728b06ee53e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.685587 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:17.698296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 982/1957] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:17.703185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523658a5-9835-43d7-a106-0728b06ee53e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.715444 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523658a5-9835-43d7-a106-0728b06ee53e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.723259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c59e3-a0a4-4ae8-9aa5-728d79b884cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.724964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c59e3-a0a4-4ae8-9aa5-728d79b884cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.725329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c59e3-a0a4-4ae8-9aa5-728d79b884cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.725797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c59e3-a0a4-4ae8-9aa5-728d79b884cc admin admin] PATH_INFO: `/v3/projects/e9253c2d51eb48dd9595756d11085376/users/47572bb1b34241cca24a6ba89f135db9/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.750158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce784f74-0ea1-4ccd-94f6-7871949b4945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.751002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c59e3-a0a4-4ae8-9aa5-728d79b884cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9253c2d51eb48dd9595756d11085376, user_id=47572bb1b34241cca24a6ba89f135db9, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.752163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce784f74-0ea1-4ccd-94f6-7871949b4945 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.754764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c59e3-a0a4-4ae8-9aa5-728d79b884cc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:17.763203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 976/1958] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:17.782040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 983/1959] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:17 2022] PUT /identity/v3/projects/e9253c2d51eb48dd9595756d11085376/users/47572bb1b34241cca24a6ba89f135db9/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:17.789715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ed399-aea2-41da-b67f-ab1709a5db9b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.789996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ed399-aea2-41da-b67f-ab1709a5db9b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.790280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ed399-aea2-41da-b67f-ab1709a5db9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.820480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-583ed399-aea2-41da-b67f-ab1709a5db9b None None] MFA Rules not processed for user `47572bb1b34241cca24a6ba89f135db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:17.821743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-583ed399-aea2-41da-b67f-ab1709a5db9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:17.902305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 977/1960] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:17.906981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c59e3-a0a4-4ae8-9aa5-728d79b884cc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:17.987129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c59e3-a0a4-4ae8-9aa5-728d79b884cc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:17.992177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a6dc9-80e1-421b-8c3c-9800391f33fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:17.992860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a6dc9-80e1-421b-8c3c-9800391f33fa admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:17.993154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a6dc9-80e1-421b-8c3c-9800391f33fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:17.993439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a6dc9-80e1-421b-8c3c-9800391f33fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:17.994162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a6dc9-80e1-421b-8c3c-9800391f33fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:17.996319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a6dc9-80e1-421b-8c3c-9800391f33fa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:18.027626 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 984/1961] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:17 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:18.030865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ed399-aea2-41da-b67f-ab1709a5db9b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:18.070362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ed399-aea2-41da-b67f-ab1709a5db9b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:18.169991 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07ec2-5d24-41ff-b61e-681a946cfc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:18.170765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07ec2-5d24-41ff-b61e-681a946cfc21 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:18.171122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07ec2-5d24-41ff-b61e-681a946cfc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:18.172943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07ec2-5d24-41ff-b61e-681a946cfc21 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:18.172943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b07ec2-5d24-41ff-b61e-681a946cfc21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:18.174417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b07ec2-5d24-41ff-b61e-681a946cfc21 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:18.196347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 978/1962] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:18 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:18.200296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a6dc9-80e1-421b-8c3c-9800391f33fa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:18.211514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a6dc9-80e1-421b-8c3c-9800391f33fa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:18.217064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a4440-e1e8-4cd4-b5f7-5fe6199a8fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:18.217935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4440-e1e8-4cd4-b5f7-5fe6199a8fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:18.218296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4440-e1e8-4cd4-b5f7-5fe6199a8fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:18.218637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4440-e1e8-4cd4-b5f7-5fe6199a8fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:18.219678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a4440-e1e8-4cd4-b5f7-5fe6199a8fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:18.221750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a4440-e1e8-4cd4-b5f7-5fe6199a8fdc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:18.237704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 985/1963] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:18.238953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07ec2-5d24-41ff-b61e-681a946cfc21 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:18.250280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07ec2-5d24-41ff-b61e-681a946cfc21 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:18.255193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda46263-d8cd-4cf0-a1c6-bdab089ca955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:18.256284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda46263-d8cd-4cf0-a1c6-bdab089ca955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:18.256573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda46263-d8cd-4cf0-a1c6-bdab089ca955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:18.256909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda46263-d8cd-4cf0-a1c6-bdab089ca955 admin admin] PATH_INFO: `/v3/projects/16aa48a577b44a5cb8212c4424175276/users/206b441f26a44daa8f08082a60a8e646/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:18.269557 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:18.273205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda46263-d8cd-4cf0-a1c6-bdab089ca955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16aa48a577b44a5cb8212c4424175276, user_id=206b441f26a44daa8f08082a60a8e646, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:18.276119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda46263-d8cd-4cf0-a1c6-bdab089ca955 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:18.295277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 979/1964] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:18 2022] PUT /identity/v3/projects/16aa48a577b44a5cb8212c4424175276/users/206b441f26a44daa8f08082a60a8e646/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:18.296344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a4440-e1e8-4cd4-b5f7-5fe6199a8fdc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:18.315644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a4440-e1e8-4cd4-b5f7-5fe6199a8fdc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:18.375638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65812878-84f3-4e60-bb83-0651c712ba0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:18.376354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65812878-84f3-4e60-bb83-0651c712ba0e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:18.376645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65812878-84f3-4e60-bb83-0651c712ba0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:18.376922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65812878-84f3-4e60-bb83-0651c712ba0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:18.377679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65812878-84f3-4e60-bb83-0651c712ba0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:18.379316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65812878-84f3-4e60-bb83-0651c712ba0e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:18.389324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 986/1965] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:18.393651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda46263-d8cd-4cf0-a1c6-bdab089ca955 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:18.404590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda46263-d8cd-4cf0-a1c6-bdab089ca955 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:18.410667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be03a68-9e94-41f6-8904-47e90180793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:18.411602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be03a68-9e94-41f6-8904-47e90180793a admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:18.411927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be03a68-9e94-41f6-8904-47e90180793a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:18.412143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be03a68-9e94-41f6-8904-47e90180793a admin admin] PATH_INFO: `/v3/projects/16aa48a577b44a5cb8212c4424175276/users/206b441f26a44daa8f08082a60a8e646/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:18.414114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be03a68-9e94-41f6-8904-47e90180793a admin admin] RBAC: Authorizing `identity:create_grant(project_id=16aa48a577b44a5cb8212c4424175276, user_id=206b441f26a44daa8f08082a60a8e646, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:18.417110 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be03a68-9e94-41f6-8904-47e90180793a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:18.440915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 980/1966] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:18 2022] PUT /identity/v3/projects/16aa48a577b44a5cb8212c4424175276/users/206b441f26a44daa8f08082a60a8e646/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:18.451632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5c149-c2d7-4ea0-afe2-ab409404c1ed None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:18.451930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5c149-c2d7-4ea0-afe2-ab409404c1ed None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:18.452250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5c149-c2d7-4ea0-afe2-ab409404c1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:18.486386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-60c5c149-c2d7-4ea0-afe2-ab409404c1ed None None] MFA Rules not processed for user `206b441f26a44daa8f08082a60a8e646`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:18.487924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-60c5c149-c2d7-4ea0-afe2-ab409404c1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:18.518898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:18.560014 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 987/1967] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:18.571831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be03a68-9e94-41f6-8904-47e90180793a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:18.628530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be03a68-9e94-41f6-8904-47e90180793a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:18.632888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b72cd3-5188-4f5c-aee8-22b4548fe3ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:18.633573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b72cd3-5188-4f5c-aee8-22b4548fe3ee service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:18.633832 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b72cd3-5188-4f5c-aee8-22b4548fe3ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:18.634114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b72cd3-5188-4f5c-aee8-22b4548fe3ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:18.692230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b72cd3-5188-4f5c-aee8-22b4548fe3ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:18.693978 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b72cd3-5188-4f5c-aee8-22b4548fe3ee service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:18.702954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 981/1968] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:20.238607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5c149-c2d7-4ea0-afe2-ab409404c1ed None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:20.248313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5c149-c2d7-4ea0-afe2-ab409404c1ed None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:20.252070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2cac3-8421-4a54-b650-e5f2236691d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:20.252738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2cac3-8421-4a54-b650-e5f2236691d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:20.253003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2cac3-8421-4a54-b650-e5f2236691d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:20.253274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2cac3-8421-4a54-b650-e5f2236691d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:20.313875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2cac3-8421-4a54-b650-e5f2236691d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:20.315369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2cac3-8421-4a54-b650-e5f2236691d4 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:20.326125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 988/1969] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:20.766174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b72cd3-5188-4f5c-aee8-22b4548fe3ee service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:20.767734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-12b72cd3-5188-4f5c-aee8-22b4548fe3ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:20.862519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b72cd3-5188-4f5c-aee8-22b4548fe3ee service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:20.882023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:20.882985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:20.883824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:20.884113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] PATH_INFO: `/v3/users/cdfeb15aa05c4809aed9a899d9d2ee1f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:20.885512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfeb15aa05c4809aed9a899d9d2ee1f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:20.887850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:20.986684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:21.008667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2cac3-8421-4a54-b650-e5f2236691d4 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:21.017118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfeb15aa05c4809aed9a899d9d2ee1f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.019072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2cac3-8421-4a54-b650-e5f2236691d4 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:21.020656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfeb15aa05c4809aed9a899d9d2ee1f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.025622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1f759-96e7-45e7-958b-9cd8be391fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:21.026284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1f759-96e7-45e7-958b-9cd8be391fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:21.026574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1f759-96e7-45e7-958b-9cd8be391fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:21.026851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1f759-96e7-45e7-958b-9cd8be391fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:21.032516 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-19e1f759-96e7-45e7-958b-9cd8be391fa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:21.055876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfeb15aa05c4809aed9a899d9d2ee1f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.064439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1f759-96e7-45e7-958b-9cd8be391fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:21.066886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1f759-96e7-45e7-958b-9cd8be391fa7 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:21.073861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfeb15aa05c4809aed9a899d9d2ee1f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.074628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfeb15aa05c4809aed9a899d9d2ee1f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.088355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 989/1970] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:21.110097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 982/1971] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:20 2022] DELETE /identity/v3/users/cdfeb15aa05c4809aed9a899d9d2ee1f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:21.114818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80260e38-4185-434c-93f6-04e84544adb2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:21.115122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80260e38-4185-434c-93f6-04e84544adb2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:21.115411 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80260e38-4185-434c-93f6-04e84544adb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:21.154909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-80260e38-4185-434c-93f6-04e84544adb2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:21.160996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-80260e38-4185-434c-93f6-04e84544adb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:21.273567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 990/1972] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:21.285379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:21.287731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:21.376177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4d2d5-248e-4175-9581-39004fea9ac5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:21.380103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43c3e9-0e3c-49af-a7be-8de489c4648e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:21.380981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43c3e9-0e3c-49af-a7be-8de489c4648e service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:21.381207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43c3e9-0e3c-49af-a7be-8de489c4648e service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:21.381459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43c3e9-0e3c-49af-a7be-8de489c4648e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:21.403859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43c3e9-0e3c-49af-a7be-8de489c4648e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:21.408023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43c3e9-0e3c-49af-a7be-8de489c4648e service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:21.437984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 983/1973] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:21.528872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80260e38-4185-434c-93f6-04e84544adb2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:21.531311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-80260e38-4185-434c-93f6-04e84544adb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:21.547149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80260e38-4185-434c-93f6-04e84544adb2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:21.573422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:21.574924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:21.575411 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:21.575980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] PATH_INFO: `/v3/users/755a639319e74b5483a6333ca458290e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:21.577500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755a639319e74b5483a6333ca458290e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:21.579911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:21.653317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755a639319e74b5483a6333ca458290e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.654044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755a639319e74b5483a6333ca458290e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.655399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:21.692897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755a639319e74b5483a6333ca458290e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.703312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755a639319e74b5483a6333ca458290e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.716120 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755a639319e74b5483a6333ca458290e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:21.728765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 991/1974] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:21 2022] DELETE /identity/v3/users/755a639319e74b5483a6333ca458290e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:21.731870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e14ef9-229b-488f-a91c-60a85e4ec1d9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:21.732171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e14ef9-229b-488f-a91c-60a85e4ec1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:21.732459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e14ef9-229b-488f-a91c-60a85e4ec1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:21.777117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-78e14ef9-229b-488f-a91c-60a85e4ec1d9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:21.778375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78e14ef9-229b-488f-a91c-60a85e4ec1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:21.819729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:21.821498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:21.868453 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 984/1975] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:21.877688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e14ef9-229b-488f-a91c-60a85e4ec1d9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:21.879272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78e14ef9-229b-488f-a91c-60a85e4ec1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:21.932107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38488f80-4fd1-4057-bf99-817bdeecb458 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:21.938033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:21.939599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:21.939987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:21.940161 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] PATH_INFO: `/v3/projects/25ffcc0b6bcd4ba2b02faf9c538cc66c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:21.942862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ffcc0b6bcd4ba2b02faf9c538cc66c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:21.945015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:21.964986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e14ef9-229b-488f-a91c-60a85e4ec1d9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:21.969421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed925a1-84f9-4376-934b-5f297e057db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:21.970485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed925a1-84f9-4376-934b-5f297e057db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:21.971241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed925a1-84f9-4376-934b-5f297e057db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:21.971634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed925a1-84f9-4376-934b-5f297e057db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:21.984727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed925a1-84f9-4376-934b-5f297e057db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:21.986638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed925a1-84f9-4376-934b-5f297e057db5 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:21.992279 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:22.003965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 985/1976] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:22.004438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ffcc0b6bcd4ba2b02faf9c538cc66c', 'request_id': 'req-38302b69-0da7-4ed1-b17b-c9b24834d196'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.015833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ffcc0b6bcd4ba2b02faf9c538cc66c', 'request_id': 'req-38302b69-0da7-4ed1-b17b-c9b24834d196'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.023381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ffcc0b6bcd4ba2b02faf9c538cc66c', 'request_id': 'req-38302b69-0da7-4ed1-b17b-c9b24834d196'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.036779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] The token cache is being invalidate because project 25ffcc0b6bcd4ba2b02faf9c538cc66c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:22.037118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.044029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.048005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 992/1977] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:21 2022] DELETE /identity/v3/projects/25ffcc0b6bcd4ba2b02faf9c538cc66c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:22.050815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed925a1-84f9-4376-934b-5f297e057db5 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:22.068665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed925a1-84f9-4376-934b-5f297e057db5 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:22.138785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:22.139672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:22.139970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:22.140251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] PATH_INFO: `/v3/users/387c1bed64b046b9a4c89437ea5916bf` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:22.141577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] RBAC: Authorizing `identity:delete_user(user_id=387c1bed64b046b9a4c89437ea5916bf)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:22.143523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:22.217159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '387c1bed64b046b9a4c89437ea5916bf'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.218494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '387c1bed64b046b9a4c89437ea5916bf'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.231690 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '387c1bed64b046b9a4c89437ea5916bf'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.243150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '387c1bed64b046b9a4c89437ea5916bf'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.244164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '387c1bed64b046b9a4c89437ea5916bf'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.262642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 986/1978] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:22 2022] DELETE /identity/v3/users/387c1bed64b046b9a4c89437ea5916bf => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:22.470640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:22.472111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:22.497441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38302b69-0da7-4ed1-b17b-c9b24834d196 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:22.620843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:22.632092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:22.635683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:22.636291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:22.636742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] PATH_INFO: `/v3/projects/25e501dce17d46b4bb53c0a6782da171` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:22.644380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e501dce17d46b4bb53c0a6782da171)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:22.646917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:22.745402 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e501dce17d46b4bb53c0a6782da171', 'request_id': 'req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.762750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e501dce17d46b4bb53c0a6782da171', 'request_id': 'req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.785827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e501dce17d46b4bb53c0a6782da171', 'request_id': 'req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.847765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] The token cache is being invalidate because project 25e501dce17d46b4bb53c0a6782da171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:22.848448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.849877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:22.853256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 993/1979] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:22 2022] DELETE /identity/v3/projects/25e501dce17d46b4bb53c0a6782da171 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:22.857268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:22.858757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:22.879918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4f9d-0ca9-4555-b94c-1299fd50842a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:23.019249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:23.020673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.021057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.021242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] PATH_INFO: `/v3/users/d714affe44e344c293e85647a9134b01` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.022748 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d714affe44e344c293e85647a9134b01)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:23.025211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:23.049689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:23.090912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd714affe44e344c293e85647a9134b01'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.091660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd714affe44e344c293e85647a9134b01'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.112231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd714affe44e344c293e85647a9134b01'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.125242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd714affe44e344c293e85647a9134b01'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.125242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f74dfd20-92e3-466b-8cd2-0ac59bfbdd7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd714affe44e344c293e85647a9134b01'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.126817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:23.128577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:23.145175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ec600-c17b-44ad-87a8-1d05f0e9da58 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:23.153301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 987/1980] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:22 2022] DELETE /identity/v3/users/d714affe44e344c293e85647a9134b01 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:23.161748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:23.163326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.168562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.169597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] PATH_INFO: `/v3/projects/6795d04f455e424dac3a848d9b0419cc` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.172722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6795d04f455e424dac3a848d9b0419cc)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:23.178287 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:23.278611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6795d04f455e424dac3a848d9b0419cc', 'request_id': 'req-70492226-4c76-48f8-8ef4-86ab96e3aa35'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.291774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6795d04f455e424dac3a848d9b0419cc', 'request_id': 'req-70492226-4c76-48f8-8ef4-86ab96e3aa35'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.301502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6795d04f455e424dac3a848d9b0419cc', 'request_id': 'req-70492226-4c76-48f8-8ef4-86ab96e3aa35'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.326639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] The token cache is being invalidate because project 6795d04f455e424dac3a848d9b0419cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:23.327198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.328575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-70492226-4c76-48f8-8ef4-86ab96e3aa35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.330749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 994/1981] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:23 2022] DELETE /identity/v3/projects/6795d04f455e424dac3a848d9b0419cc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:23.337643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f65918-9e23-41fc-b70b-9fab79904152 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.338810 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f65918-9e23-41fc-b70b-9fab79904152 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.339494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f65918-9e23-41fc-b70b-9fab79904152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.374037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d5f65918-9e23-41fc-b70b-9fab79904152 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:23.377269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d5f65918-9e23-41fc-b70b-9fab79904152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:23.461837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 988/1982] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:23.470707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65654cf-3573-4eff-96e8-60142a4c43ae None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.471821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65654cf-3573-4eff-96e8-60142a4c43ae None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.472712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65654cf-3573-4eff-96e8-60142a4c43ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.498495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b65654cf-3573-4eff-96e8-60142a4c43ae None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:23.500293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b65654cf-3573-4eff-96e8-60142a4c43ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:23.505866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f65918-9e23-41fc-b70b-9fab79904152 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:23.509018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d5f65918-9e23-41fc-b70b-9fab79904152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:23.517007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 995/1983] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:23.521315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65654cf-3573-4eff-96e8-60142a4c43ae None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:23.531919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65654cf-3573-4eff-96e8-60142a4c43ae None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:23.537345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ec762-589e-44f4-80a7-5a9e06c6c1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:23.538178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ec762-589e-44f4-80a7-5a9e06c6c1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.538531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ec762-589e-44f4-80a7-5a9e06c6c1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.538728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f65918-9e23-41fc-b70b-9fab79904152 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:23.538922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ec762-589e-44f4-80a7-5a9e06c6c1c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.540084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ec762-589e-44f4-80a7-5a9e06c6c1c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:23.540084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ec762-589e-44f4-80a7-5a9e06c6c1c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:23.541871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ec762-589e-44f4-80a7-5a9e06c6c1c7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:23.545399 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:23.546246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.546602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.546888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] PATH_INFO: `/v3/projects/651f065924734074a73e3d2e7093f303` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.548315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=651f065924734074a73e3d2e7093f303)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:23.550992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:23.560159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 996/1984] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:42:23.561081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ec762-589e-44f4-80a7-5a9e06c6c1c7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:23.571665 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ec762-589e-44f4-80a7-5a9e06c6c1c7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:23.576611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f554a-5fb3-4095-952f-2db70d1935fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:23.577372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f554a-5fb3-4095-952f-2db70d1935fe admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.577809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f554a-5fb3-4095-952f-2db70d1935fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.578076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f554a-5fb3-4095-952f-2db70d1935fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.580025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f554a-5fb3-4095-952f-2db70d1935fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:23.581110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f554a-5fb3-4095-952f-2db70d1935fe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:23.618596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 997/1985] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:23 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:23.625153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f554a-5fb3-4095-952f-2db70d1935fe admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:23.637640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '651f065924734074a73e3d2e7093f303', 'request_id': 'req-5be51265-dbd8-452e-a72e-588cbd131144'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.645397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '651f065924734074a73e3d2e7093f303', 'request_id': 'req-5be51265-dbd8-452e-a72e-588cbd131144'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.652421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '651f065924734074a73e3d2e7093f303', 'request_id': 'req-5be51265-dbd8-452e-a72e-588cbd131144'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.662387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:23.668740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] The token cache is being invalidate because project 651f065924734074a73e3d2e7093f303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:23.669038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.669860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be51265-dbd8-452e-a72e-588cbd131144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:23.672699 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 989/1986] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:23 2022] DELETE /identity/v3/projects/651f065924734074a73e3d2e7093f303 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:23.677850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2c2b2-7936-44b9-9e73-78aa3bed6d58 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.678411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2c2b2-7936-44b9-9e73-78aa3bed6d58 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.678682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2c2b2-7936-44b9-9e73-78aa3bed6d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.701967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-28b2c2b2-7936-44b9-9e73-78aa3bed6d58 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:23.703293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-28b2c2b2-7936-44b9-9e73-78aa3bed6d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:23.703566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f554a-5fb3-4095-952f-2db70d1935fe admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:23.719052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:23.767551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fb26f-c12a-480f-8355-450264b076ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:23.768255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fb26f-c12a-480f-8355-450264b076ce admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.768542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fb26f-c12a-480f-8355-450264b076ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.768939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fb26f-c12a-480f-8355-450264b076ce admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.769498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fb26f-c12a-480f-8355-450264b076ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:23.771638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fb26f-c12a-480f-8355-450264b076ce admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:23.777294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 990/1987] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:23.786377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53923da1-48f2-430f-8860-471e466fd135 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.786753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53923da1-48f2-430f-8860-471e466fd135 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.787125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53923da1-48f2-430f-8860-471e466fd135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.795354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 998/1988] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:23 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:23.799177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fb26f-c12a-480f-8355-450264b076ce admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:23.801483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e09fb26f-c12a-480f-8355-450264b076ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:23.809842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-53923da1-48f2-430f-8860-471e466fd135 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:23.811049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-53923da1-48f2-430f-8860-471e466fd135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:23.814296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fb26f-c12a-480f-8355-450264b076ce admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:23.821590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec63ed-0d62-4539-b7b6-456122e826bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:23.822378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec63ed-0d62-4539-b7b6-456122e826bc admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.822766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec63ed-0d62-4539-b7b6-456122e826bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.823346 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec63ed-0d62-4539-b7b6-456122e826bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.824207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec63ed-0d62-4539-b7b6-456122e826bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:23.824818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 991/1989] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:23.826160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec63ed-0d62-4539-b7b6-456122e826bc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:23.828953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53923da1-48f2-430f-8860-471e466fd135 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:23.849591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 999/1990] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:23.854698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec63ed-0d62-4539-b7b6-456122e826bc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:23.875590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53923da1-48f2-430f-8860-471e466fd135 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:23.876143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec63ed-0d62-4539-b7b6-456122e826bc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:23.885930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a4860-4ea2-4224-9dd6-0f02870d5a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:23.891936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc6d23-fba1-4949-a402-d4a258d7a78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:23.893729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc6d23-fba1-4949-a402-d4a258d7a78d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.897050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc6d23-fba1-4949-a402-d4a258d7a78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.897357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc6d23-fba1-4949-a402-d4a258d7a78d admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.898127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc6d23-fba1-4949-a402-d4a258d7a78d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:23.898453 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc6d23-fba1-4949-a402-d4a258d7a78d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:23.900746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc6d23-fba1-4949-a402-d4a258d7a78d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:23.903630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a4860-4ea2-4224-9dd6-0f02870d5a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:23.903940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a4860-4ea2-4224-9dd6-0f02870d5a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:23.905191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a4860-4ea2-4224-9dd6-0f02870d5a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:23.908609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a4860-4ea2-4224-9dd6-0f02870d5a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:23.912932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a4860-4ea2-4224-9dd6-0f02870d5a0e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:23.955947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1000/1991] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:23.963517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 992/1992] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:42:23.964611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a4860-4ea2-4224-9dd6-0f02870d5a0e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:23.967760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc6d23-fba1-4949-a402-d4a258d7a78d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.007262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc6d23-fba1-4949-a402-d4a258d7a78d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.012050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a4860-4ea2-4224-9dd6-0f02870d5a0e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.020309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30581213-8db1-401e-9198-630b84cf31ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.021250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581213-8db1-401e-9198-630b84cf31ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.021545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581213-8db1-401e-9198-630b84cf31ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.021830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581213-8db1-401e-9198-630b84cf31ca admin admin] PATH_INFO: `/v3/projects/510dbbad4c4242e4a3a162aefcb220c4/users/30cd2edc2f0747ceb062d2f9404c8ad1/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.033645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdebb0-40ee-4a6e-9941-a8a6b229265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.034449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdebb0-40ee-4a6e-9941-a8a6b229265e admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.034737 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdebb0-40ee-4a6e-9941-a8a6b229265e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.035019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdebb0-40ee-4a6e-9941-a8a6b229265e admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.048128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30581213-8db1-401e-9198-630b84cf31ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=510dbbad4c4242e4a3a162aefcb220c4, user_id=30cd2edc2f0747ceb062d2f9404c8ad1, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.050545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30581213-8db1-401e-9198-630b84cf31ca admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.051095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdebb0-40ee-4a6e-9941-a8a6b229265e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.055750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdebb0-40ee-4a6e-9941-a8a6b229265e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.107679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1001/1993] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:23 2022] PUT /identity/v3/projects/510dbbad4c4242e4a3a162aefcb220c4/users/30cd2edc2f0747ceb062d2f9404c8ad1/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:24.111703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95eaf0-023a-4b6d-95bb-63749121198f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.112016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95eaf0-023a-4b6d-95bb-63749121198f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.112330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95eaf0-023a-4b6d-95bb-63749121198f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.123338 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 993/1994] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:23 2022] POST /identity/v3/projects => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:24.127264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdebb0-40ee-4a6e-9941-a8a6b229265e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.206342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6f95eaf0-023a-4b6d-95bb-63749121198f None None] MFA Rules not processed for user `30cd2edc2f0747ceb062d2f9404c8ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:24.206342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6f95eaf0-023a-4b6d-95bb-63749121198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:24.314892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdebb0-40ee-4a6e-9941-a8a6b229265e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.325483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76792d57-9c79-40bc-aeb3-5683fa63ae92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.326188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76792d57-9c79-40bc-aeb3-5683fa63ae92 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.326478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76792d57-9c79-40bc-aeb3-5683fa63ae92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.327339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76792d57-9c79-40bc-aeb3-5683fa63ae92 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.327450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76792d57-9c79-40bc-aeb3-5683fa63ae92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.329498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76792d57-9c79-40bc-aeb3-5683fa63ae92 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.333994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1002/1995] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:24.339073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95eaf0-023a-4b6d-95bb-63749121198f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.359896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95eaf0-023a-4b6d-95bb-63749121198f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.366997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05a52a-4977-4d29-a28d-c591cfee564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.367186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 994/1996] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:24 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:42:24.367281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05a52a-4977-4d29-a28d-c591cfee564d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.367580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05a52a-4977-4d29-a28d-c591cfee564d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.367918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05a52a-4977-4d29-a28d-c591cfee564d admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.368631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05a52a-4977-4d29-a28d-c591cfee564d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.371600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05a52a-4977-4d29-a28d-c591cfee564d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.373092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76792d57-9c79-40bc-aeb3-5683fa63ae92 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.376550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:24.400806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76792d57-9c79-40bc-aeb3-5683fa63ae92 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.410373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1003/1997] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:24 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:24.410809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22672bf-f7e4-4240-8be5-612ed9ac48cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.411277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05a52a-4977-4d29-a28d-c591cfee564d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.411548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22672bf-f7e4-4240-8be5-612ed9ac48cc admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.411838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22672bf-f7e4-4240-8be5-612ed9ac48cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.412180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22672bf-f7e4-4240-8be5-612ed9ac48cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.412872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22672bf-f7e4-4240-8be5-612ed9ac48cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.414506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22672bf-f7e4-4240-8be5-612ed9ac48cc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.433893 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 995/1998] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:24.440732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22672bf-f7e4-4240-8be5-612ed9ac48cc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.464953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22672bf-f7e4-4240-8be5-612ed9ac48cc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.502837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05a52a-4977-4d29-a28d-c591cfee564d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.515559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ef667-e8ef-4489-9045-39eaf106aecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.516563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ef667-e8ef-4489-9045-39eaf106aecb admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.516916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ef667-e8ef-4489-9045-39eaf106aecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.517204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ef667-e8ef-4489-9045-39eaf106aecb admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.517988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ef667-e8ef-4489-9045-39eaf106aecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.521375 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ef667-e8ef-4489-9045-39eaf106aecb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.560955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1004/1999] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:24 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:24.563480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ef667-e8ef-4489-9045-39eaf106aecb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.577235 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ef667-e8ef-4489-9045-39eaf106aecb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.584537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:24.584849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987b6de-89ce-4666-98c7-e3c43ed9afa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.586432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b6de-89ce-4666-98c7-e3c43ed9afa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.587162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b6de-89ce-4666-98c7-e3c43ed9afa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.587678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b6de-89ce-4666-98c7-e3c43ed9afa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.588600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987b6de-89ce-4666-98c7-e3c43ed9afa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.590720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987b6de-89ce-4666-98c7-e3c43ed9afa4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.596720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da92849-d0db-49ed-9349-4c6c4aecb4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.597626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da92849-d0db-49ed-9349-4c6c4aecb4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.597894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da92849-d0db-49ed-9349-4c6c4aecb4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.598244 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da92849-d0db-49ed-9349-4c6c4aecb4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.600187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da92849-d0db-49ed-9349-4c6c4aecb4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.607516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da92849-d0db-49ed-9349-4c6c4aecb4a8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.613844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1005/2000] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:24.617037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987b6de-89ce-4666-98c7-e3c43ed9afa4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.638674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 996/2001] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:42:24.640722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987b6de-89ce-4666-98c7-e3c43ed9afa4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.642893 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da92849-d0db-49ed-9349-4c6c4aecb4a8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.646010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f41d0-91be-4482-ade3-aace7c0e7642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.647024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f41d0-91be-4482-ade3-aace7c0e7642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.647477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f41d0-91be-4482-ade3-aace7c0e7642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.647837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f41d0-91be-4482-ade3-aace7c0e7642 admin admin] PATH_INFO: `/v3/projects/59c522e2ff0b404396a81cca1fca8ae9/users/d11a138aaf92409596c45ea067bc1a62/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.660446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da92849-d0db-49ed-9349-4c6c4aecb4a8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.666731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d189-b7bd-49b3-a3a9-64639b54a3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.667826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d189-b7bd-49b3-a3a9-64639b54a3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.668210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d189-b7bd-49b3-a3a9-64639b54a3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.668592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d189-b7bd-49b3-a3a9-64639b54a3c2 admin admin] PATH_INFO: `/v3/projects/2711f4094ea54967b09d459af8135fe7/users/92d0be4f89ba4ab5850b6f33f533bc9e/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.685924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d189-b7bd-49b3-a3a9-64639b54a3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2711f4094ea54967b09d459af8135fe7, user_id=92d0be4f89ba4ab5850b6f33f533bc9e, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.688939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d189-b7bd-49b3-a3a9-64639b54a3c2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.690124 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f41d0-91be-4482-ade3-aace7c0e7642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c522e2ff0b404396a81cca1fca8ae9, user_id=d11a138aaf92409596c45ea067bc1a62, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.695061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f41d0-91be-4482-ade3-aace7c0e7642 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.705310 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 997/2002] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:24 2022] PUT /identity/v3/projects/2711f4094ea54967b09d459af8135fe7/users/92d0be4f89ba4ab5850b6f33f533bc9e/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:24.710637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b8f87-3f63-4ec3-b421-01bbec20b359 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.711064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b8f87-3f63-4ec3-b421-01bbec20b359 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.711445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b8f87-3f63-4ec3-b421-01bbec20b359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.725823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1006/2003] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:24 2022] PUT /identity/v3/projects/59c522e2ff0b404396a81cca1fca8ae9/users/d11a138aaf92409596c45ea067bc1a62/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:24.730100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f41d0-91be-4482-ade3-aace7c0e7642 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.731432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dd7b8f87-3f63-4ec3-b421-01bbec20b359 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:24.732792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dd7b8f87-3f63-4ec3-b421-01bbec20b359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:24.829771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f41d0-91be-4482-ade3-aace7c0e7642 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.837211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 998/2004] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:24.838480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e83192-e5af-4756-ba0d-d39990c3ea4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.839212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83192-e5af-4756-ba0d-d39990c3ea4a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.839544 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83192-e5af-4756-ba0d-d39990c3ea4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.839836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83192-e5af-4756-ba0d-d39990c3ea4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.840591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e83192-e5af-4756-ba0d-d39990c3ea4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.842262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e83192-e5af-4756-ba0d-d39990c3ea4a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.852675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b8f87-3f63-4ec3-b421-01bbec20b359 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.859812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1007/2005] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:24.861365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dd7b8f87-3f63-4ec3-b421-01bbec20b359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:24.863847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e83192-e5af-4756-ba0d-d39990c3ea4a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:24.882376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e83192-e5af-4756-ba0d-d39990c3ea4a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.892104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dd777-e247-48a3-9fc6-d879e25f3157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:24.894336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dd777-e247-48a3-9fc6-d879e25f3157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.894780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dd777-e247-48a3-9fc6-d879e25f3157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.895322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dd777-e247-48a3-9fc6-d879e25f3157 admin admin] PATH_INFO: `/v3/projects/59c522e2ff0b404396a81cca1fca8ae9/users/d11a138aaf92409596c45ea067bc1a62/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.899033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dd777-e247-48a3-9fc6-d879e25f3157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c522e2ff0b404396a81cca1fca8ae9, user_id=d11a138aaf92409596c45ea067bc1a62, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:24.907868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dd777-e247-48a3-9fc6-d879e25f3157 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:24.924609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1008/2006] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:24 2022] PUT /identity/v3/projects/59c522e2ff0b404396a81cca1fca8ae9/users/d11a138aaf92409596c45ea067bc1a62/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:24.948319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c16b0-df18-489c-afee-939e96648388 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:24.948679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c16b0-df18-489c-afee-939e96648388 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:24.949153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c16b0-df18-489c-afee-939e96648388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:24.953861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b8f87-3f63-4ec3-b421-01bbec20b359 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:24.990282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ef7c16b0-df18-489c-afee-939e96648388 None None] MFA Rules not processed for user `d11a138aaf92409596c45ea067bc1a62`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:24.991802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ef7c16b0-df18-489c-afee-939e96648388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:25.003216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:25.019643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3dfc2-6fb9-40fb-8e5c-12d55182c3dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:25.020287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3dfc2-6fb9-40fb-8e5c-12d55182c3dd service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:25.020576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3dfc2-6fb9-40fb-8e5c-12d55182c3dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:25.020889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3dfc2-6fb9-40fb-8e5c-12d55182c3dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:25.067901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:25.081922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1009/2007] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:42:25.093307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c16b0-df18-489c-afee-939e96648388 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:25.095312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ef7c16b0-df18-489c-afee-939e96648388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:25.096804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3dfc2-6fb9-40fb-8e5c-12d55182c3dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:25.098272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3dfc2-6fb9-40fb-8e5c-12d55182c3dd service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:25.109114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 999/2008] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:25.178278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c16b0-df18-489c-afee-939e96648388 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:25.184172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a2d07-c287-445c-96a7-f522cbf3b2b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:25.185305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a2d07-c287-445c-96a7-f522cbf3b2b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:25.185697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a2d07-c287-445c-96a7-f522cbf3b2b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:25.186170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a2d07-c287-445c-96a7-f522cbf3b2b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:25.260360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a2d07-c287-445c-96a7-f522cbf3b2b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:25.261953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a2d07-c287-445c-96a7-f522cbf3b2b9 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:25.275966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1010/2009] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:25.750297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3dfc2-6fb9-40fb-8e5c-12d55182c3dd service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:25.751899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-18b3dfc2-6fb9-40fb-8e5c-12d55182c3dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:25.782471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3dfc2-6fb9-40fb-8e5c-12d55182c3dd service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:25.787389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:25.788339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:25.788615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:25.788894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] PATH_INFO: `/v3/users/07b8ae99f8b14168aaa490d299272389` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:25.796417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07b8ae99f8b14168aaa490d299272389)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:25.798521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:25.885361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07b8ae99f8b14168aaa490d299272389'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:25.886183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07b8ae99f8b14168aaa490d299272389'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:25.900385 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07b8ae99f8b14168aaa490d299272389'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:25.909306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07b8ae99f8b14168aaa490d299272389'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:25.911590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07b8ae99f8b14168aaa490d299272389'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:25.938207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1000/2010] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:25 2022] DELETE /identity/v3/users/07b8ae99f8b14168aaa490d299272389 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:25.944831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247494-5b74-42c0-90db-fd4379a6a83a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:25.945362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247494-5b74-42c0-90db-fd4379a6a83a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:25.945531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247494-5b74-42c0-90db-fd4379a6a83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:25.980245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-27247494-5b74-42c0-90db-fd4379a6a83a None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:25.983059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-27247494-5b74-42c0-90db-fd4379a6a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:26.086923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1011/2011] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:26.100667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:26.102769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:26.189774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c3f93-f438-44f5-b50b-c8599244f922 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:26.194153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d1f8c-61ad-4942-ab55-6220b3861850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:26.195028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d1f8c-61ad-4942-ab55-6220b3861850 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.196258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d1f8c-61ad-4942-ab55-6220b3861850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.196681 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d1f8c-61ad-4942-ab55-6220b3861850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.209606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d1f8c-61ad-4942-ab55-6220b3861850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:26.211485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d1f8c-61ad-4942-ab55-6220b3861850 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:26.223204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1001/2012] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:26.269521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4223e4-6998-4183-8f4f-b9c62d0388c7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.269910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4223e4-6998-4183-8f4f-b9c62d0388c7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.270842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4223e4-6998-4183-8f4f-b9c62d0388c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.276859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:26.301889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ed4223e4-6998-4183-8f4f-b9c62d0388c7 None None] MFA Rules not processed for user `92d0be4f89ba4ab5850b6f33f533bc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:26.302985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ed4223e4-6998-4183-8f4f-b9c62d0388c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:26.404238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1012/2013] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:26.409873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d1f8c-61ad-4942-ab55-6220b3861850 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:26.411905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-928d1f8c-61ad-4942-ab55-6220b3861850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:26.443199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d1f8c-61ad-4942-ab55-6220b3861850 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:26.458482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85b4a7-f304-4483-9b9b-d908e5846bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:26.459185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85b4a7-f304-4483-9b9b-d908e5846bed admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.460418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85b4a7-f304-4483-9b9b-d908e5846bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.460418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85b4a7-f304-4483-9b9b-d908e5846bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.461643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85b4a7-f304-4483-9b9b-d908e5846bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:26.464885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85b4a7-f304-4483-9b9b-d908e5846bed admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:26.510097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1002/2014] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:26 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:26.513862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4223e4-6998-4183-8f4f-b9c62d0388c7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:26.530155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4223e4-6998-4183-8f4f-b9c62d0388c7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:26.586916 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85b4a7-f304-4483-9b9b-d908e5846bed admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:26.592519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5f85b4a7-f304-4483-9b9b-d908e5846bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:26.616799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3edae-9957-4d6a-aba3-a3c6674c9891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:26.617465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3edae-9957-4d6a-aba3-a3c6674c9891 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.617806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3edae-9957-4d6a-aba3-a3c6674c9891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.618093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3edae-9957-4d6a-aba3-a3c6674c9891 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.618799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3edae-9957-4d6a-aba3-a3c6674c9891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:26.621706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3edae-9957-4d6a-aba3-a3c6674c9891 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:26.634264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:26.650490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1013/2015] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:26 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:26.653822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3edae-9957-4d6a-aba3-a3c6674c9891 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:26.665252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3edae-9957-4d6a-aba3-a3c6674c9891 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:26.670775 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958b436-b7a0-49dc-92ca-47d07cda5e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:26.671502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958b436-b7a0-49dc-92ca-47d07cda5e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.671786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958b436-b7a0-49dc-92ca-47d07cda5e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.672260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958b436-b7a0-49dc-92ca-47d07cda5e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.673008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958b436-b7a0-49dc-92ca-47d07cda5e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:26.674635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958b436-b7a0-49dc-92ca-47d07cda5e3a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:26.681423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85b4a7-f304-4483-9b9b-d908e5846bed admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:26.685961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1014/2016] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:26.687757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:26.688646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.688935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.689218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] PATH_INFO: `/v3/projects/898873b33a414016bb843230d1a18486` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.692532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=898873b33a414016bb843230d1a18486)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:26.694875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:26.697491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958b436-b7a0-49dc-92ca-47d07cda5e3a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:26.711953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958b436-b7a0-49dc-92ca-47d07cda5e3a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:26.718254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70fbee-1854-45fe-afa1-e1790ebe5e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:26.719450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70fbee-1854-45fe-afa1-e1790ebe5e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.719789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70fbee-1854-45fe-afa1-e1790ebe5e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.720088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70fbee-1854-45fe-afa1-e1790ebe5e4e admin admin] PATH_INFO: `/v3/projects/ac5bffbbbe684494b20cdb9eea4146b2/users/93e98220c1b94c968d7533758df51eab/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.739671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70fbee-1854-45fe-afa1-e1790ebe5e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5bffbbbe684494b20cdb9eea4146b2, user_id=93e98220c1b94c968d7533758df51eab, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:26.740060 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70fbee-1854-45fe-afa1-e1790ebe5e4e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:26.755835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '898873b33a414016bb843230d1a18486', 'request_id': 'req-2082394f-b416-473b-bd30-9de27bf751bd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:26.756062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1015/2017] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:26 2022] PUT /identity/v3/projects/ac5bffbbbe684494b20cdb9eea4146b2/users/93e98220c1b94c968d7533758df51eab/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:26.759809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70fbee-1854-45fe-afa1-e1790ebe5e4e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:26.762235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '898873b33a414016bb843230d1a18486', 'request_id': 'req-2082394f-b416-473b-bd30-9de27bf751bd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:26.770156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '898873b33a414016bb843230d1a18486', 'request_id': 'req-2082394f-b416-473b-bd30-9de27bf751bd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:26.791344 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] The token cache is being invalidate because project 898873b33a414016bb843230d1a18486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:26.791686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:26.792533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:26.798045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1003/2018] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:26 2022] DELETE /identity/v3/projects/898873b33a414016bb843230d1a18486 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:26.799106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:26.800570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:26.820734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:26.840452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70fbee-1854-45fe-afa1-e1790ebe5e4e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:26.845492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e3382-9870-457f-8f9e-501dabf0a983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:26.846177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e3382-9870-457f-8f9e-501dabf0a983 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.847353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e3382-9870-457f-8f9e-501dabf0a983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.847353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e3382-9870-457f-8f9e-501dabf0a983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.848011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e3382-9870-457f-8f9e-501dabf0a983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:26.850085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e3382-9870-457f-8f9e-501dabf0a983 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:26.863514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1016/2019] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:26.866623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e3382-9870-457f-8f9e-501dabf0a983 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:26.868618 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-575e3382-9870-457f-8f9e-501dabf0a983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:26.889249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e3382-9870-457f-8f9e-501dabf0a983 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:26.889385 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082394f-b416-473b-bd30-9de27bf751bd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:26.894519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:26.894662 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cae9a-7ffb-4c49-8baf-697bed1cbe1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:26.895687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.895808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cae9a-7ffb-4c49-8baf-697bed1cbe1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:26.896164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cae9a-7ffb-4c49-8baf-697bed1cbe1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.896524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:26.896645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cae9a-7ffb-4c49-8baf-697bed1cbe1f admin admin] PATH_INFO: `/v3/projects/ac5bffbbbe684494b20cdb9eea4146b2/users/93e98220c1b94c968d7533758df51eab/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.896996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] PATH_INFO: `/v3/users/47572bb1b34241cca24a6ba89f135db9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:26.898314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47572bb1b34241cca24a6ba89f135db9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:26.898911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cae9a-7ffb-4c49-8baf-697bed1cbe1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5bffbbbe684494b20cdb9eea4146b2, user_id=93e98220c1b94c968d7533758df51eab, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:26.900708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:26.902080 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cae9a-7ffb-4c49-8baf-697bed1cbe1f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:26.914418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1017/2020] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:26 2022] PUT /identity/v3/projects/ac5bffbbbe684494b20cdb9eea4146b2/users/93e98220c1b94c968d7533758df51eab/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:26.960820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47572bb1b34241cca24a6ba89f135db9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:26.961620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47572bb1b34241cca24a6ba89f135db9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:26.972142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:26.978066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47572bb1b34241cca24a6ba89f135db9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:26.986746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47572bb1b34241cca24a6ba89f135db9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:26.987585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47572bb1b34241cca24a6ba89f135db9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:27.002687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1004/2021] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:26 2022] DELETE /identity/v3/users/47572bb1b34241cca24a6ba89f135db9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:27.367096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cae9a-7ffb-4c49-8baf-697bed1cbe1f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:27.369865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e58cae9a-7ffb-4c49-8baf-697bed1cbe1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:27.538068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cae9a-7ffb-4c49-8baf-697bed1cbe1f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:27.643578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:27.644580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:27.644864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:27.645168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] PATH_INFO: `/v3/projects/e9253c2d51eb48dd9595756d11085376` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:27.646632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9253c2d51eb48dd9595756d11085376)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:27.648946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:27.720106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9253c2d51eb48dd9595756d11085376', 'request_id': 'req-685035c7-8619-4422-b31d-1e509b4c25a2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:27.732293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9253c2d51eb48dd9595756d11085376', 'request_id': 'req-685035c7-8619-4422-b31d-1e509b4c25a2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:27.741269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9253c2d51eb48dd9595756d11085376', 'request_id': 'req-685035c7-8619-4422-b31d-1e509b4c25a2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:27.763615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] The token cache is being invalidate because project e9253c2d51eb48dd9595756d11085376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:27.764023 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:27.765396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-685035c7-8619-4422-b31d-1e509b4c25a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:27.767631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1018/2022] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:27 2022] DELETE /identity/v3/projects/e9253c2d51eb48dd9595756d11085376 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:27.772785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:27.788844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0ad1e-3627-4ce6-aa54-65a50dbb0df4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:27.845956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49503c83-72d3-4ccb-888f-29ccab7352f2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:27.847369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49503c83-72d3-4ccb-888f-29ccab7352f2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:27.848244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49503c83-72d3-4ccb-888f-29ccab7352f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:27.879825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:27.886009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:27.887926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:27.888518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:27.888960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] PATH_INFO: `/v3/users/206b441f26a44daa8f08082a60a8e646` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:27.890827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=206b441f26a44daa8f08082a60a8e646)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:27.893390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:27.934113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-49503c83-72d3-4ccb-888f-29ccab7352f2 None None] MFA Rules not processed for user `93e98220c1b94c968d7533758df51eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:27.935786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-49503c83-72d3-4ccb-888f-29ccab7352f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:27.982292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206b441f26a44daa8f08082a60a8e646'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:27.983620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206b441f26a44daa8f08082a60a8e646'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.019143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206b441f26a44daa8f08082a60a8e646'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.036173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206b441f26a44daa8f08082a60a8e646'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.037749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206b441f26a44daa8f08082a60a8e646'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.077993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1005/2023] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:27 2022] DELETE /identity/v3/users/206b441f26a44daa8f08082a60a8e646 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:28.080111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:28.082824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:28.137372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1019/2024] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:28.150764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49503c83-72d3-4ccb-888f-29ccab7352f2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:28.154111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-49503c83-72d3-4ccb-888f-29ccab7352f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:28.210443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:28.217572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da67650-83b4-459b-9fb4-f6fe3be574f0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:28.231037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:28.233904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:28.234232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:28.234579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] PATH_INFO: `/v3/users/30cd2edc2f0747ceb062d2f9404c8ad1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:28.236887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cd2edc2f0747ceb062d2f9404c8ad1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:28.238945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:28.256259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49503c83-72d3-4ccb-888f-29ccab7352f2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:28.296331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cd2edc2f0747ceb062d2f9404c8ad1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.297416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cd2edc2f0747ceb062d2f9404c8ad1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.315765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cd2edc2f0747ceb062d2f9404c8ad1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.327437 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:28.328446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cd2edc2f0747ceb062d2f9404c8ad1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.329296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:28.329478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dfd75af9-64b1-4006-9055-9b7bd3150599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cd2edc2f0747ceb062d2f9404c8ad1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.329879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:28.330338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:28.333160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:28.353482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1006/2025] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:28 2022] DELETE /identity/v3/users/30cd2edc2f0747ceb062d2f9404c8ad1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:28.358698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000b1bd-55d3-4d45-9b76-9a644c963dcf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:28.359140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000b1bd-55d3-4d45-9b76-9a644c963dcf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:28.359570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000b1bd-55d3-4d45-9b76-9a644c963dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:28.400454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1000b1bd-55d3-4d45-9b76-9a644c963dcf None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:28.402177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1000b1bd-55d3-4d45-9b76-9a644c963dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:28.475719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:28.477396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:28.486064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1007/2026] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:28.494134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1020/2027] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:28.497942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000b1bd-55d3-4d45-9b76-9a644c963dcf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:28.499549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1000b1bd-55d3-4d45-9b76-9a644c963dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:28.538291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:28.540319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:28.549050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:28.573130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441d380-122c-476b-b08d-76a71b0fbb74 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:28.582530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000b1bd-55d3-4d45-9b76-9a644c963dcf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:28.583644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:28.584702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:28.584993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:28.585365 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] PATH_INFO: `/v3/projects/16aa48a577b44a5cb8212c4424175276` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:28.586380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a561-4ca4-4c19-9fa6-320e7c9536c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:28.586758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16aa48a577b44a5cb8212c4424175276)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:28.587034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a561-4ca4-4c19-9fa6-320e7c9536c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:28.587318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a561-4ca4-4c19-9fa6-320e7c9536c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:28.587626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a561-4ca4-4c19-9fa6-320e7c9536c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:28.588903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:28.600383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a561-4ca4-4c19-9fa6-320e7c9536c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:28.602100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a561-4ca4-4c19-9fa6-320e7c9536c5 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:28.634292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1008/2028] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:28.660067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16aa48a577b44a5cb8212c4424175276', 'request_id': 'req-adc30e0b-9a7d-4142-a818-d75de564d3e0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.672076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16aa48a577b44a5cb8212c4424175276', 'request_id': 'req-adc30e0b-9a7d-4142-a818-d75de564d3e0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.684197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16aa48a577b44a5cb8212c4424175276', 'request_id': 'req-adc30e0b-9a7d-4142-a818-d75de564d3e0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.699615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] The token cache is being invalidate because project 16aa48a577b44a5cb8212c4424175276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:28.700075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.700691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-adc30e0b-9a7d-4142-a818-d75de564d3e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:28.703008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1021/2029] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:28 2022] DELETE /identity/v3/projects/16aa48a577b44a5cb8212c4424175276 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:28.708736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3efef-f7aa-443a-9874-37a769d78e2f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:28.709125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3efef-f7aa-443a-9874-37a769d78e2f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:28.709524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3efef-f7aa-443a-9874-37a769d78e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:28.730181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8fa3efef-f7aa-443a-9874-37a769d78e2f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:28.731193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8fa3efef-f7aa-443a-9874-37a769d78e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:28.747886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:28.835396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1009/2030] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:28.844210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5517480-9012-4356-b578-ea7c7d568197 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:28.844547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5517480-9012-4356-b578-ea7c7d568197 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:28.844950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5517480-9012-4356-b578-ea7c7d568197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:28.887920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c5517480-9012-4356-b578-ea7c7d568197 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:28.891958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c5517480-9012-4356-b578-ea7c7d568197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:28.912906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1022/2031] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:28.917916 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3efef-f7aa-443a-9874-37a769d78e2f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:28.935897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3efef-f7aa-443a-9874-37a769d78e2f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:28.941025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442e26d-aee3-4005-94dc-5f899947500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:28.941823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442e26d-aee3-4005-94dc-5f899947500a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:28.942508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442e26d-aee3-4005-94dc-5f899947500a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:28.942881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442e26d-aee3-4005-94dc-5f899947500a admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:28.943665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442e26d-aee3-4005-94dc-5f899947500a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:28.944107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442e26d-aee3-4005-94dc-5f899947500a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:28.945770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442e26d-aee3-4005-94dc-5f899947500a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:28.964807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1010/2032] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:42:28.966140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5517480-9012-4356-b578-ea7c7d568197 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:28.977959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5517480-9012-4356-b578-ea7c7d568197 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:28.983538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb5eef-465e-4372-a621-6336ec0b40ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:28.983820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb5eef-465e-4372-a621-6336ec0b40ee admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:28.984651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb5eef-465e-4372-a621-6336ec0b40ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:28.984947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb5eef-465e-4372-a621-6336ec0b40ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:28.985745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb5eef-465e-4372-a621-6336ec0b40ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:28.988303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb5eef-465e-4372-a621-6336ec0b40ee admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:29.015442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1023/2033] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:28 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:29.020139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442e26d-aee3-4005-94dc-5f899947500a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:29.030770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442e26d-aee3-4005-94dc-5f899947500a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:29.097834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a92f9-6ce8-43be-997e-0b4570ed20d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:29.098650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a92f9-6ce8-43be-997e-0b4570ed20d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:29.098943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a92f9-6ce8-43be-997e-0b4570ed20d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:29.099224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a92f9-6ce8-43be-997e-0b4570ed20d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:29.108217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a92f9-6ce8-43be-997e-0b4570ed20d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:29.110886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a92f9-6ce8-43be-997e-0b4570ed20d8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:29.146280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1011/2034] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:29 2022] POST /identity/v3/users => generated 284 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:29.150987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb5eef-465e-4372-a621-6336ec0b40ee admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:29.171293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb5eef-465e-4372-a621-6336ec0b40ee admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:29.176688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e95b2-a3c3-469f-a12e-96a7c76262a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:29.177465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e95b2-a3c3-469f-a12e-96a7c76262a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:29.177826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e95b2-a3c3-469f-a12e-96a7c76262a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:29.178309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e95b2-a3c3-469f-a12e-96a7c76262a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:29.179119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e95b2-a3c3-469f-a12e-96a7c76262a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:29.180857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e95b2-a3c3-469f-a12e-96a7c76262a7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:29.207495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1024/2035] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:29.209376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a92f9-6ce8-43be-997e-0b4570ed20d8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:29.219784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a92f9-6ce8-43be-997e-0b4570ed20d8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:29.225117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61adbc0-8e93-43ee-bbdb-d6923d411abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:29.225860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61adbc0-8e93-43ee-bbdb-d6923d411abe admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:29.226150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61adbc0-8e93-43ee-bbdb-d6923d411abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:29.226431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61adbc0-8e93-43ee-bbdb-d6923d411abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:29.227165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61adbc0-8e93-43ee-bbdb-d6923d411abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:29.229498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61adbc0-8e93-43ee-bbdb-d6923d411abe admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:29.250239 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1012/2036] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:29.254943 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e95b2-a3c3-469f-a12e-96a7c76262a7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:29.259147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61adbc0-8e93-43ee-bbdb-d6923d411abe admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:29.261342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f61adbc0-8e93-43ee-bbdb-d6923d411abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:29.267089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e95b2-a3c3-469f-a12e-96a7c76262a7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:29.272370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c91f54-68ac-40e5-8409-6be4dbd12586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:29.277207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c91f54-68ac-40e5-8409-6be4dbd12586 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:29.278103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c91f54-68ac-40e5-8409-6be4dbd12586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:29.278431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c91f54-68ac-40e5-8409-6be4dbd12586 admin admin] PATH_INFO: `/v3/projects/4f631df7309b47e184f487903e91b38d/users/85c2b9f4ee6a4419980c1649de7c6a55/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:29.287855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61adbc0-8e93-43ee-bbdb-d6923d411abe admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:29.293666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:29.294559 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:29.294941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:29.295333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] PATH_INFO: `/v3/projects/510dbbad4c4242e4a3a162aefcb220c4` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:29.297656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=510dbbad4c4242e4a3a162aefcb220c4)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:29.300579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:29.301070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c91f54-68ac-40e5-8409-6be4dbd12586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f631df7309b47e184f487903e91b38d, user_id=85c2b9f4ee6a4419980c1649de7c6a55, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:29.303845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c91f54-68ac-40e5-8409-6be4dbd12586 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:29.323711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1025/2037] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:29 2022] PUT /identity/v3/projects/4f631df7309b47e184f487903e91b38d/users/85c2b9f4ee6a4419980c1649de7c6a55/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:29.336051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bc6c4-1a94-48ad-b8a3-5e435c0a2dae None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:29.336349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bc6c4-1a94-48ad-b8a3-5e435c0a2dae None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:29.336637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bc6c4-1a94-48ad-b8a3-5e435c0a2dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:29.359081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510dbbad4c4242e4a3a162aefcb220c4', 'request_id': 'req-20bea75f-8200-473b-b4df-6abdb0643cbd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.365107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-656bc6c4-1a94-48ad-b8a3-5e435c0a2dae None None] MFA Rules not processed for user `85c2b9f4ee6a4419980c1649de7c6a55`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:29.366220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510dbbad4c4242e4a3a162aefcb220c4', 'request_id': 'req-20bea75f-8200-473b-b4df-6abdb0643cbd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.366361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-656bc6c4-1a94-48ad-b8a3-5e435c0a2dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:29.372211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510dbbad4c4242e4a3a162aefcb220c4', 'request_id': 'req-20bea75f-8200-473b-b4df-6abdb0643cbd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.385637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] The token cache is being invalidate because project 510dbbad4c4242e4a3a162aefcb220c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:29.386039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.386685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.388329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1013/2038] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:29 2022] DELETE /identity/v3/projects/510dbbad4c4242e4a3a162aefcb220c4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:29.392438 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:29.393905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:29.405030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:29.435732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1026/2039] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:29.451560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bc6c4-1a94-48ad-b8a3-5e435c0a2dae None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:29.454626 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-656bc6c4-1a94-48ad-b8a3-5e435c0a2dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:29.468046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bea75f-8200-473b-b4df-6abdb0643cbd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:29.468284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:29.473362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:29.474649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:29.475052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:29.475428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] PATH_INFO: `/v3/users/d11a138aaf92409596c45ea067bc1a62` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:29.476681 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d11a138aaf92409596c45ea067bc1a62)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:29.478905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:29.527878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd11a138aaf92409596c45ea067bc1a62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.528701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd11a138aaf92409596c45ea067bc1a62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.539673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bc6c4-1a94-48ad-b8a3-5e435c0a2dae None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:29.543285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd11a138aaf92409596c45ea067bc1a62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.544196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc21faf2237d405bba03e5309d93629f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:29.545438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:29.545718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:29.546003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:29.548094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:29.550425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd11a138aaf92409596c45ea067bc1a62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.551142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd11a138aaf92409596c45ea067bc1a62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:29.559865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:29.561432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:29.565629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1014/2040] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:29 2022] DELETE /identity/v3/users/d11a138aaf92409596c45ea067bc1a62 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:29.572596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1027/2041] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:29.865840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:29.867280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:29.943216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aeaa7d-37b5-4ba8-ba1e-7825078b1703 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:29.948712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:29.949539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:29.949831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:29.950115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] PATH_INFO: `/v3/projects/59c522e2ff0b404396a81cca1fca8ae9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:29.951312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c522e2ff0b404396a81cca1fca8ae9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:29.953665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:29.984357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:29.986220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:29.996226 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:30.012735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c522e2ff0b404396a81cca1fca8ae9', 'request_id': 'req-0265ab51-c83f-42d8-be39-96b2875d56ed'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:30.019928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c522e2ff0b404396a81cca1fca8ae9', 'request_id': 'req-0265ab51-c83f-42d8-be39-96b2875d56ed'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:30.026547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c522e2ff0b404396a81cca1fca8ae9', 'request_id': 'req-0265ab51-c83f-42d8-be39-96b2875d56ed'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:30.055785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] The token cache is being invalidate because project 59c522e2ff0b404396a81cca1fca8ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:30.056142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:30.056939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0265ab51-c83f-42d8-be39-96b2875d56ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:30.059218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4809825-4852-445a-9309-2b472c0b4851 service glance] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:30.064999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1015/2042] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:29 2022] DELETE /identity/v3/projects/59c522e2ff0b404396a81cca1fca8ae9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:30.163643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01106c-f51b-49b3-9594-c0b4b4affd5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:30.164328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01106c-f51b-49b3-9594-c0b4b4affd5a service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:30.164615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01106c-f51b-49b3-9594-c0b4b4affd5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:30.164890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01106c-f51b-49b3-9594-c0b4b4affd5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:30.167094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7d01106c-f51b-49b3-9594-c0b4b4affd5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:30.255314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01106c-f51b-49b3-9594-c0b4b4affd5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:30.256877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01106c-f51b-49b3-9594-c0b4b4affd5a service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:30.262671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:30.272971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1028/2043] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:40.048468 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=13,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:43.189948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf78a33-20c1-422f-8c4b-17c8985a1f03 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:43.190638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf78a33-20c1-422f-8c4b-17c8985a1f03 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:43.190638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf78a33-20c1-422f-8c4b-17c8985a1f03 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:43.192477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1016/2044] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:42:43.225653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b490e3-71b1-4e02-b11b-cd6570091cab None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:43.226257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b490e3-71b1-4e02-b11b-cd6570091cab None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:43.226257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b490e3-71b1-4e02-b11b-cd6570091cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:43.247345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e4b490e3-71b1-4e02-b11b-cd6570091cab None None] MFA Rules not processed for user `cbafea9ed6494a3f9aeba4140a32de4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:43.248736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e4b490e3-71b1-4e02-b11b-cd6570091cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:43.321222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1029/2045] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:42:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:43.336119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf78a33-20c1-422f-8c4b-17c8985a1f03 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:43.338213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ecf78a33-20c1-422f-8c4b-17c8985a1f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:43.358736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf78a33-20c1-422f-8c4b-17c8985a1f03 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:43.364955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b04152-3461-41dc-b204-c4c4fec57810 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:43.365213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b04152-3461-41dc-b204-c4c4fec57810 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:43.365297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b04152-3461-41dc-b204-c4c4fec57810 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:43.365698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b04152-3461-41dc-b204-c4c4fec57810 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:43.376621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b04152-3461-41dc-b204-c4c4fec57810 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:43.378419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b04152-3461-41dc-b204-c4c4fec57810 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:43.389682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1017/2046] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:43.912334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099645b-65f5-4c4c-ba14-c44e0184098f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:43.912651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099645b-65f5-4c4c-ba14-c44e0184098f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:43.912935 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099645b-65f5-4c4c-ba14-c44e0184098f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:43.914379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1030/2047] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:42:43.947466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdb740-13a7-45b0-bd3f-942af8e03c61 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:43.947795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdb740-13a7-45b0-bd3f-942af8e03c61 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:43.948119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdb740-13a7-45b0-bd3f-942af8e03c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:43.970919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-70cdb740-13a7-45b0-bd3f-942af8e03c61 None None] MFA Rules not processed for user `cbafea9ed6494a3f9aeba4140a32de4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:43.972890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-70cdb740-13a7-45b0-bd3f-942af8e03c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:43.978852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099645b-65f5-4c4c-ba14-c44e0184098f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:43.982433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3099645b-65f5-4c4c-ba14-c44e0184098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:43.983368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1018/2048] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:42:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:44.004212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdb740-13a7-45b0-bd3f-942af8e03c61 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:44.022288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdb740-13a7-45b0-bd3f-942af8e03c61 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:44.031732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62364cb-dd45-4972-b066-772bac76fb25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:44.033005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62364cb-dd45-4972-b066-772bac76fb25 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:44.033750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62364cb-dd45-4972-b066-772bac76fb25 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:44.034434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62364cb-dd45-4972-b066-772bac76fb25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:44.050806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62364cb-dd45-4972-b066-772bac76fb25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:44.052643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62364cb-dd45-4972-b066-772bac76fb25 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:44.067610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1019/2049] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:42:44.098714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099645b-65f5-4c4c-ba14-c44e0184098f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:44.103810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:44.105224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:44.105516 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:44.105796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] PATH_INFO: `/v3/users/92d0be4f89ba4ab5850b6f33f533bc9e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:44.107015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=92d0be4f89ba4ab5850b6f33f533bc9e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:44.109041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:44.158921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92d0be4f89ba4ab5850b6f33f533bc9e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.159913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92d0be4f89ba4ab5850b6f33f533bc9e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.175501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92d0be4f89ba4ab5850b6f33f533bc9e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.182934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92d0be4f89ba4ab5850b6f33f533bc9e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.192543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92d0be4f89ba4ab5850b6f33f533bc9e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.199980 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1031/2050] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:43 2022] DELETE /identity/v3/users/92d0be4f89ba4ab5850b6f33f533bc9e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:44.381930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62364cb-dd45-4972-b066-772bac76fb25 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:44.392970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62364cb-dd45-4972-b066-772bac76fb25 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:44.465445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:44.466433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:44.466781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:44.467119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] PATH_INFO: `/v3/projects/2711f4094ea54967b09d459af8135fe7` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:44.468553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2711f4094ea54967b09d459af8135fe7)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:44.470523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:44.519662 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2711f4094ea54967b09d459af8135fe7', 'request_id': 'req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.526469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2711f4094ea54967b09d459af8135fe7', 'request_id': 'req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.533484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2711f4094ea54967b09d459af8135fe7', 'request_id': 'req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.547017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] The token cache is being invalidate because project 2711f4094ea54967b09d459af8135fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:44.547311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.547996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.549584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1020/2051] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:44 2022] DELETE /identity/v3/projects/2711f4094ea54967b09d459af8135fe7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:44.553824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:44.555241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:44.570049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63917b36-ee3e-4ada-9ac3-e040a1a17bad admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:44.583957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:44.638247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:44.639257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:44.639600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:44.640086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] PATH_INFO: `/v3/users/93e98220c1b94c968d7533758df51eab` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:44.641428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93e98220c1b94c968d7533758df51eab)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:44.643683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:44.701380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93e98220c1b94c968d7533758df51eab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.702169 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93e98220c1b94c968d7533758df51eab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.714508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93e98220c1b94c968d7533758df51eab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.721971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93e98220c1b94c968d7533758df51eab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.730139 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5940543c-6959-4e8f-835f-84b6a37da6c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93e98220c1b94c968d7533758df51eab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:44.738422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1032/2052] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:42:44 2022] DELETE /identity/v3/users/93e98220c1b94c968d7533758df51eab => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:42:44.920744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:44.922865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:44.937200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975f16c-48ae-4e7d-ac1f-bd1f4ea85325 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:44.992560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:45.000016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:45.001482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.001851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.002433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] PATH_INFO: `/v3/projects/ac5bffbbbe684494b20cdb9eea4146b2` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.003855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac5bffbbbe684494b20cdb9eea4146b2)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:45.005985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:45.061443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac5bffbbbe684494b20cdb9eea4146b2', 'request_id': 'req-5a0927fd-05de-410d-86b4-eb231edba91d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:45.070394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac5bffbbbe684494b20cdb9eea4146b2', 'request_id': 'req-5a0927fd-05de-410d-86b4-eb231edba91d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:45.077305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac5bffbbbe684494b20cdb9eea4146b2', 'request_id': 'req-5a0927fd-05de-410d-86b4-eb231edba91d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:45.093396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] The token cache is being invalidate because project ac5bffbbbe684494b20cdb9eea4146b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:42:45.093957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:45.094831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5a0927fd-05de-410d-86b4-eb231edba91d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:42:45.097097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1021/2053] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:42:44 2022] DELETE /identity/v3/projects/ac5bffbbbe684494b20cdb9eea4146b2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:45.107426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3afa9-26f2-4255-8e10-14e1bc613522 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.107426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3afa9-26f2-4255-8e10-14e1bc613522 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.107426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3afa9-26f2-4255-8e10-14e1bc613522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.121810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1ea3afa9-26f2-4255-8e10-14e1bc613522 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:45.123015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1ea3afa9-26f2-4255-8e10-14e1bc613522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:45.180473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1033/2054] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:45.188549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb575a7-6ab7-4353-8c90-a89e2fc2ca9e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.188844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb575a7-6ab7-4353-8c90-a89e2fc2ca9e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.189097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb575a7-6ab7-4353-8c90-a89e2fc2ca9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.211309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-abb575a7-6ab7-4353-8c90-a89e2fc2ca9e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:45.212704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-abb575a7-6ab7-4353-8c90-a89e2fc2ca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:45.221952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1022/2055] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:45.226312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3afa9-26f2-4255-8e10-14e1bc613522 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:45.236580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3afa9-26f2-4255-8e10-14e1bc613522 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:45.241571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb26e77-f97f-4778-b30d-ee30537ed125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:45.242251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb26e77-f97f-4778-b30d-ee30537ed125 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.242541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb26e77-f97f-4778-b30d-ee30537ed125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.242826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb26e77-f97f-4778-b30d-ee30537ed125 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.243595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb26e77-f97f-4778-b30d-ee30537ed125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:42:45.243911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb26e77-f97f-4778-b30d-ee30537ed125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:45.245540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb26e77-f97f-4778-b30d-ee30537ed125 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:45.258501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1034/2056] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:42:45.263024 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb575a7-6ab7-4353-8c90-a89e2fc2ca9e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:45.276715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb575a7-6ab7-4353-8c90-a89e2fc2ca9e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:45.283383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e92a8c-dde2-4e88-884b-70c0ece7af5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:45.284167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e92a8c-dde2-4e88-884b-70c0ece7af5f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.284518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e92a8c-dde2-4e88-884b-70c0ece7af5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.284861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e92a8c-dde2-4e88-884b-70c0ece7af5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.285567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e92a8c-dde2-4e88-884b-70c0ece7af5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:45.287502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e92a8c-dde2-4e88-884b-70c0ece7af5f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:45.321018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1023/2057] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:42:45 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:45.322539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb26e77-f97f-4778-b30d-ee30537ed125 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:45.333480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb26e77-f97f-4778-b30d-ee30537ed125 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:45.354458 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:45.405325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c168a-fbfc-4a80-a211-ad4634b06309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:45.406076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c168a-fbfc-4a80-a211-ad4634b06309 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.406359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c168a-fbfc-4a80-a211-ad4634b06309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.406640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c168a-fbfc-4a80-a211-ad4634b06309 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.407330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c168a-fbfc-4a80-a211-ad4634b06309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:45.409329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c168a-fbfc-4a80-a211-ad4634b06309 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:45.433446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1035/2058] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:42:45 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:42:45.437540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e92a8c-dde2-4e88-884b-70c0ece7af5f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:45.448692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e92a8c-dde2-4e88-884b-70c0ece7af5f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:45.453825 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250902b0-7256-43fb-af3f-3574852d1f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:45.454518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250902b0-7256-43fb-af3f-3574852d1f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.454787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250902b0-7256-43fb-af3f-3574852d1f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.455068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250902b0-7256-43fb-af3f-3574852d1f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.455812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250902b0-7256-43fb-af3f-3574852d1f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:45.457466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250902b0-7256-43fb-af3f-3574852d1f57 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:45.467877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1024/2059] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:45.471981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c168a-fbfc-4a80-a211-ad4634b06309 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:45.482195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c168a-fbfc-4a80-a211-ad4634b06309 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:45.487432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9d66c-c035-4832-9f8b-5daf3abb1bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:45.488087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9d66c-c035-4832-9f8b-5daf3abb1bab admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.488366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9d66c-c035-4832-9f8b-5daf3abb1bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.488636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9d66c-c035-4832-9f8b-5daf3abb1bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.489372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9d66c-c035-4832-9f8b-5daf3abb1bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:45.491016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9d66c-c035-4832-9f8b-5daf3abb1bab admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:45.501816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1036/2060] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:42:45.506197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250902b0-7256-43fb-af3f-3574852d1f57 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:45.516380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250902b0-7256-43fb-af3f-3574852d1f57 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:45.521005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d6c79-8595-4cff-877a-5b0364fdbe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:45.521885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6c79-8595-4cff-877a-5b0364fdbe81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.522168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6c79-8595-4cff-877a-5b0364fdbe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.522440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6c79-8595-4cff-877a-5b0364fdbe81 admin admin] PATH_INFO: `/v3/projects/1945b77ecb2c417b89cc469276306077/users/815edf512a2f414593204a3d02780767/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.537511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d6c79-8595-4cff-877a-5b0364fdbe81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1945b77ecb2c417b89cc469276306077, user_id=815edf512a2f414593204a3d02780767, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:45.540219 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d6c79-8595-4cff-877a-5b0364fdbe81 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:45.555631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1025/2061] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:42:45 2022] PUT /identity/v3/projects/1945b77ecb2c417b89cc469276306077/users/815edf512a2f414593204a3d02780767/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:42:45.565921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b24da4-b72c-4118-b23d-9673488a35cf None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.566380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b24da4-b72c-4118-b23d-9673488a35cf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.566467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b24da4-b72c-4118-b23d-9673488a35cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.599371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d6b24da4-b72c-4118-b23d-9673488a35cf None None] MFA Rules not processed for user `815edf512a2f414593204a3d02780767`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:42:45.600646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d6b24da4-b72c-4118-b23d-9673488a35cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:45.671679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1037/2062] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:42:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:42:45.683236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d6c79-8595-4cff-877a-5b0364fdbe81 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:42:45.685357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f74d6c79-8595-4cff-877a-5b0364fdbe81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:42:45.756998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d6c79-8595-4cff-877a-5b0364fdbe81 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:42:45.762116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d81ef-ac0d-47e7-aef7-d82c593caa84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:42:45.762797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d81ef-ac0d-47e7-aef7-d82c593caa84 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:42:45.763058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d81ef-ac0d-47e7-aef7-d82c593caa84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:42:45.763341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d81ef-ac0d-47e7-aef7-d82c593caa84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:42:45.773173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d81ef-ac0d-47e7-aef7-d82c593caa84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:42:45.774620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d81ef-ac0d-47e7-aef7-d82c593caa84 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:42:45.783847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1026/2063] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:42:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:42:55.667956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:42:55.779716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:05.179896 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b24da4-b72c-4118-b23d-9673488a35cf None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:05.181416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d6b24da4-b72c-4118-b23d-9673488a35cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:05.252613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b24da4-b72c-4118-b23d-9673488a35cf None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:05.257067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:05.257962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:05.258250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:05.258532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] PATH_INFO: `/v3/users/3b4865ad7a6d43b1b13593a142c2dec1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:05.259750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b4865ad7a6d43b1b13593a142c2dec1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:05.261722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:05.310924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b4865ad7a6d43b1b13593a142c2dec1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.312030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b4865ad7a6d43b1b13593a142c2dec1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.325970 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b4865ad7a6d43b1b13593a142c2dec1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.333469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b4865ad7a6d43b1b13593a142c2dec1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.342929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1378d6e-c27f-404f-bff8-ba674cd7b489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b4865ad7a6d43b1b13593a142c2dec1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.349498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1038/2064] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:05 2022] DELETE /identity/v3/users/3b4865ad7a6d43b1b13593a142c2dec1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:05.764323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d81ef-ac0d-47e7-aef7-d82c593caa84 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:05.776296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d81ef-ac0d-47e7-aef7-d82c593caa84 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:05.846515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:05.847405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:05.847754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:05.848106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] PATH_INFO: `/v3/projects/a81ec4c4754044b58723da1ce589b86c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:05.849303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a81ec4c4754044b58723da1ce589b86c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:05.851306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:05.898260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a81ec4c4754044b58723da1ce589b86c', 'request_id': 'req-3119862b-8337-4823-a7a4-aa99c18a4d10'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.904352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a81ec4c4754044b58723da1ce589b86c', 'request_id': 'req-3119862b-8337-4823-a7a4-aa99c18a4d10'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.911990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a81ec4c4754044b58723da1ce589b86c', 'request_id': 'req-3119862b-8337-4823-a7a4-aa99c18a4d10'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.930989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] The token cache is being invalidate because project a81ec4c4754044b58723da1ce589b86c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:05.931264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.931932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3119862b-8337-4823-a7a4-aa99c18a4d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:05.933526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1027/2065] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:05 2022] DELETE /identity/v3/projects/a81ec4c4754044b58723da1ce589b86c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:05.938849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67ecfa-90d1-4eb2-81de-437951db8791 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:05.939145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67ecfa-90d1-4eb2-81de-437951db8791 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:05.939431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67ecfa-90d1-4eb2-81de-437951db8791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:05.958775 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ce67ecfa-90d1-4eb2-81de-437951db8791 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:05.960030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ce67ecfa-90d1-4eb2-81de-437951db8791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:06.022091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1039/2066] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:06.031332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688c81-5dfb-491d-adc3-8c2d4b0bfb51 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:06.031664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688c81-5dfb-491d-adc3-8c2d4b0bfb51 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:06.031951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688c81-5dfb-491d-adc3-8c2d4b0bfb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:06.051477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-24688c81-5dfb-491d-adc3-8c2d4b0bfb51 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:06.052795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-24688c81-5dfb-491d-adc3-8c2d4b0bfb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:06.061555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1028/2067] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:06.065984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67ecfa-90d1-4eb2-81de-437951db8791 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:06.079644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67ecfa-90d1-4eb2-81de-437951db8791 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:06.085438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387a4b2-7ae1-4b29-933e-4cebcfaeb99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:06.086092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387a4b2-7ae1-4b29-933e-4cebcfaeb99a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:06.086401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387a4b2-7ae1-4b29-933e-4cebcfaeb99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:06.086672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387a4b2-7ae1-4b29-933e-4cebcfaeb99a admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:06.087392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0387a4b2-7ae1-4b29-933e-4cebcfaeb99a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:43:06.087726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0387a4b2-7ae1-4b29-933e-4cebcfaeb99a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:06.089339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0387a4b2-7ae1-4b29-933e-4cebcfaeb99a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:06.100916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1040/2068] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:43:06.105217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24688c81-5dfb-491d-adc3-8c2d4b0bfb51 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:06.114683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24688c81-5dfb-491d-adc3-8c2d4b0bfb51 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:06.121055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efc939-e6de-447d-8b40-5344f8c4740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:06.121782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efc939-e6de-447d-8b40-5344f8c4740b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:06.122074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efc939-e6de-447d-8b40-5344f8c4740b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:06.122658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efc939-e6de-447d-8b40-5344f8c4740b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:06.127317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efc939-e6de-447d-8b40-5344f8c4740b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:06.129760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efc939-e6de-447d-8b40-5344f8c4740b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:06.170015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1029/2069] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:06 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:06.175542 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387a4b2-7ae1-4b29-933e-4cebcfaeb99a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:06.185674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387a4b2-7ae1-4b29-933e-4cebcfaeb99a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:06.189968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:06.250732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e227c82-473f-467b-99ae-c64f49d3ab5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:06.251418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e227c82-473f-467b-99ae-c64f49d3ab5e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:06.251734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e227c82-473f-467b-99ae-c64f49d3ab5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:06.252012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e227c82-473f-467b-99ae-c64f49d3ab5e admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:06.252704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e227c82-473f-467b-99ae-c64f49d3ab5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:06.254808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e227c82-473f-467b-99ae-c64f49d3ab5e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:06.277911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1041/2070] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:06 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:06.280702 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efc939-e6de-447d-8b40-5344f8c4740b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:06.291886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efc939-e6de-447d-8b40-5344f8c4740b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:06.297207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff7205-8813-48e5-92a9-8c48560db0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:06.297882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff7205-8813-48e5-92a9-8c48560db0ca admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:06.298202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff7205-8813-48e5-92a9-8c48560db0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:06.298486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff7205-8813-48e5-92a9-8c48560db0ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:06.299224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff7205-8813-48e5-92a9-8c48560db0ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:06.300930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff7205-8813-48e5-92a9-8c48560db0ca admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:06.313342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1030/2071] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:43:06.317546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e227c82-473f-467b-99ae-c64f49d3ab5e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:06.328299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e227c82-473f-467b-99ae-c64f49d3ab5e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:06.333512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c25cb-e5bd-483a-9ac1-eab3f029322c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:06.334187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c25cb-e5bd-483a-9ac1-eab3f029322c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:06.334475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c25cb-e5bd-483a-9ac1-eab3f029322c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:06.334753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c25cb-e5bd-483a-9ac1-eab3f029322c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:06.335557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c25cb-e5bd-483a-9ac1-eab3f029322c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:06.337186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c25cb-e5bd-483a-9ac1-eab3f029322c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:06.347162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1042/2072] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:06.351206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff7205-8813-48e5-92a9-8c48560db0ca admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:06.361207 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff7205-8813-48e5-92a9-8c48560db0ca admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:06.366180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb59d6-ff04-42a1-8419-4b75583cb2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:06.367070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb59d6-ff04-42a1-8419-4b75583cb2d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:06.367359 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb59d6-ff04-42a1-8419-4b75583cb2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:06.367703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb59d6-ff04-42a1-8419-4b75583cb2d8 admin admin] PATH_INFO: `/v3/projects/d654d9138940476c976ce7e9ffb82f3f/users/dd393deb44594fbfa21630a7bbdb5c62/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:06.382186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb59d6-ff04-42a1-8419-4b75583cb2d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d654d9138940476c976ce7e9ffb82f3f, user_id=dd393deb44594fbfa21630a7bbdb5c62, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:06.384897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb59d6-ff04-42a1-8419-4b75583cb2d8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:06.399350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1031/2073] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:06 2022] PUT /identity/v3/projects/d654d9138940476c976ce7e9ffb82f3f/users/dd393deb44594fbfa21630a7bbdb5c62/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:43:06.410059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13822abf-dbbe-49af-9114-662a9ea36c12 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:06.410384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13822abf-dbbe-49af-9114-662a9ea36c12 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:06.410722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13822abf-dbbe-49af-9114-662a9ea36c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:06.442717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-13822abf-dbbe-49af-9114-662a9ea36c12 None None] MFA Rules not processed for user `dd393deb44594fbfa21630a7bbdb5c62`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:06.444032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-13822abf-dbbe-49af-9114-662a9ea36c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:06.515238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1043/2074] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:06.526464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb59d6-ff04-42a1-8419-4b75583cb2d8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:06.528721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9beb59d6-ff04-42a1-8419-4b75583cb2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:06.561127 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:06.595862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb59d6-ff04-42a1-8419-4b75583cb2d8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:06.601634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d0ee0-1142-4bfb-abef-3bf0f12f0fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:06.602332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d0ee0-1142-4bfb-abef-3bf0f12f0fbb service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:06.602702 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d0ee0-1142-4bfb-abef-3bf0f12f0fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:06.602986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d0ee0-1142-4bfb-abef-3bf0f12f0fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:06.613719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d0ee0-1142-4bfb-abef-3bf0f12f0fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:06.618062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d0ee0-1142-4bfb-abef-3bf0f12f0fbb service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:06.628575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1032/2075] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:16.512135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:16.624220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=14 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:25.511585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13822abf-dbbe-49af-9114-662a9ea36c12 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:25.513205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-13822abf-dbbe-49af-9114-662a9ea36c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:25.587848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13822abf-dbbe-49af-9114-662a9ea36c12 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:25.593258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:25.594373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:25.594758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:25.595440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] PATH_INFO: `/v3/users/815edf512a2f414593204a3d02780767` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:25.596922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=815edf512a2f414593204a3d02780767)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:25.599085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:25.663750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815edf512a2f414593204a3d02780767'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:25.664833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815edf512a2f414593204a3d02780767'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:25.692567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815edf512a2f414593204a3d02780767'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:25.705381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815edf512a2f414593204a3d02780767'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:25.717310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815edf512a2f414593204a3d02780767'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:25.726720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1044/2076] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:25 2022] DELETE /identity/v3/users/815edf512a2f414593204a3d02780767 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:25.731114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701d7fd-b30d-4f1a-b788-ee5c76e7f3a1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:25.731976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701d7fd-b30d-4f1a-b788-ee5c76e7f3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:25.732350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701d7fd-b30d-4f1a-b788-ee5c76e7f3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:25.758323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6701d7fd-b30d-4f1a-b788-ee5c76e7f3a1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:25.759592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6701d7fd-b30d-4f1a-b788-ee5c76e7f3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:25.828198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1033/2077] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:25.839564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:25.841032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:25.927038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc0007-7539-4d36-86c2-77da5fc281f1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:25.932134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3a13d-ee68-49b1-8fde-d92297d48c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:25.932975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3a13d-ee68-49b1-8fde-d92297d48c48 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:25.933853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3a13d-ee68-49b1-8fde-d92297d48c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:25.934142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3a13d-ee68-49b1-8fde-d92297d48c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:25.947295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3a13d-ee68-49b1-8fde-d92297d48c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:25.949168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3a13d-ee68-49b1-8fde-d92297d48c48 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:25.960792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1045/2078] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:26.252340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701d7fd-b30d-4f1a-b788-ee5c76e7f3a1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:26.253782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6701d7fd-b30d-4f1a-b788-ee5c76e7f3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:26.267264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701d7fd-b30d-4f1a-b788-ee5c76e7f3a1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:26.274008 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:26.274604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.274931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.275143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] PATH_INFO: `/v3/projects/1945b77ecb2c417b89cc469276306077` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.277425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1945b77ecb2c417b89cc469276306077)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:26.279708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:26.338505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1945b77ecb2c417b89cc469276306077', 'request_id': 'req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:26.344784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1945b77ecb2c417b89cc469276306077', 'request_id': 'req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:26.350673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1945b77ecb2c417b89cc469276306077', 'request_id': 'req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:26.364683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] The token cache is being invalidate because project 1945b77ecb2c417b89cc469276306077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:26.364974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:26.365598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f1fcd0e0-b8de-4afc-9b88-b0088dc6a998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:26.367476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1034/2079] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:26 2022] DELETE /identity/v3/projects/1945b77ecb2c417b89cc469276306077 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:26.372197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e79918-5f4a-45ae-a8b3-fdfbcc09b0a2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.372507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e79918-5f4a-45ae-a8b3-fdfbcc09b0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.372792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e79918-5f4a-45ae-a8b3-fdfbcc09b0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.394138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-43e79918-5f4a-45ae-a8b3-fdfbcc09b0a2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:26.395353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-43e79918-5f4a-45ae-a8b3-fdfbcc09b0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:26.419900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:26.478761 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1046/2080] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:26.487502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcedfe4-bede-46de-91a1-e9dd10d631b7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.487994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcedfe4-bede-46de-91a1-e9dd10d631b7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.488371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcedfe4-bede-46de-91a1-e9dd10d631b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.507472 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5dcedfe4-bede-46de-91a1-e9dd10d631b7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:26.508708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5dcedfe4-bede-46de-91a1-e9dd10d631b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:26.516784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1035/2081] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:26.521007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e79918-5f4a-45ae-a8b3-fdfbcc09b0a2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:26.530655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e79918-5f4a-45ae-a8b3-fdfbcc09b0a2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:26.535365 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d1e3b-13e9-4723-92e6-f4bf7014efe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:26.536059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d1e3b-13e9-4723-92e6-f4bf7014efe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.536350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d1e3b-13e9-4723-92e6-f4bf7014efe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.536632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d1e3b-13e9-4723-92e6-f4bf7014efe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.537367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d1e3b-13e9-4723-92e6-f4bf7014efe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:43:26.538304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d1e3b-13e9-4723-92e6-f4bf7014efe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:26.539996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d1e3b-13e9-4723-92e6-f4bf7014efe6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:26.552594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1047/2082] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:43:26.556976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcedfe4-bede-46de-91a1-e9dd10d631b7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:26.567427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcedfe4-bede-46de-91a1-e9dd10d631b7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:26.573093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129429c-93cf-4e74-8c63-1f2fe18f2653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:26.573754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129429c-93cf-4e74-8c63-1f2fe18f2653 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.574038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129429c-93cf-4e74-8c63-1f2fe18f2653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.574311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129429c-93cf-4e74-8c63-1f2fe18f2653 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.575017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129429c-93cf-4e74-8c63-1f2fe18f2653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:26.577064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129429c-93cf-4e74-8c63-1f2fe18f2653 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:26.605362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1036/2083] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:26 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:43:26.609951 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d1e3b-13e9-4723-92e6-f4bf7014efe6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:26.622656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d1e3b-13e9-4723-92e6-f4bf7014efe6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:26.686204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c77bd3-33e2-487c-8a2f-82c462669d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:26.686806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c77bd3-33e2-487c-8a2f-82c462669d4e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.687090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c77bd3-33e2-487c-8a2f-82c462669d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.687363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c77bd3-33e2-487c-8a2f-82c462669d4e admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.688088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c77bd3-33e2-487c-8a2f-82c462669d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:26.690010 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c77bd3-33e2-487c-8a2f-82c462669d4e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:26.717104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1048/2084] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:26 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:26.718247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129429c-93cf-4e74-8c63-1f2fe18f2653 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:26.728597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129429c-93cf-4e74-8c63-1f2fe18f2653 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:26.733422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608309c-deb9-43d3-8915-028381fc66ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:26.734061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608309c-deb9-43d3-8915-028381fc66ca admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.734353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608309c-deb9-43d3-8915-028381fc66ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.734629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608309c-deb9-43d3-8915-028381fc66ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.735355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608309c-deb9-43d3-8915-028381fc66ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:26.736971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608309c-deb9-43d3-8915-028381fc66ca admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:26.751362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1037/2085] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:26.752710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c77bd3-33e2-487c-8a2f-82c462669d4e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:26.763879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c77bd3-33e2-487c-8a2f-82c462669d4e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:26.769867 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3da63-a616-4b15-bd99-909dd5e21da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:26.769999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3da63-a616-4b15-bd99-909dd5e21da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.770178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3da63-a616-4b15-bd99-909dd5e21da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.770453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3da63-a616-4b15-bd99-909dd5e21da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.771183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3da63-a616-4b15-bd99-909dd5e21da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:26.772819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3da63-a616-4b15-bd99-909dd5e21da0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:26.786863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1049/2086] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:43:26.788291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608309c-deb9-43d3-8915-028381fc66ca admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:26.798467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608309c-deb9-43d3-8915-028381fc66ca admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:26.804162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0f79c-b8f9-4fda-b294-79dd7979e0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:26.805284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0f79c-b8f9-4fda-b294-79dd7979e0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.805868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0f79c-b8f9-4fda-b294-79dd7979e0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.805868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0f79c-b8f9-4fda-b294-79dd7979e0ae admin admin] PATH_INFO: `/v3/projects/a0edf440dd4e413da0db9a65d2c3a592/users/bdcd1fa6d703402283a1ae45107970a7/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.821788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0f79c-b8f9-4fda-b294-79dd7979e0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0edf440dd4e413da0db9a65d2c3a592, user_id=bdcd1fa6d703402283a1ae45107970a7, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:26.824488 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0f79c-b8f9-4fda-b294-79dd7979e0ae admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:26.838569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1038/2087] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:26 2022] PUT /identity/v3/projects/a0edf440dd4e413da0db9a65d2c3a592/users/bdcd1fa6d703402283a1ae45107970a7/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:26.849012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0737a-7246-4101-a377-1923e76de3a2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:26.849314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0737a-7246-4101-a377-1923e76de3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:26.849599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0737a-7246-4101-a377-1923e76de3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:26.876959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-20f0737a-7246-4101-a377-1923e76de3a2 None None] MFA Rules not processed for user `bdcd1fa6d703402283a1ae45107970a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:26.878181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-20f0737a-7246-4101-a377-1923e76de3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:26.955969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1050/2088] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:26.961336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0f79c-b8f9-4fda-b294-79dd7979e0ae admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:26.984344 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:27.028664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0f79c-b8f9-4fda-b294-79dd7979e0ae admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.033403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650a3d5-98ce-4f1e-b542-7296217526cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.034067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650a3d5-98ce-4f1e-b542-7296217526cf admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.034353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650a3d5-98ce-4f1e-b542-7296217526cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.034629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650a3d5-98ce-4f1e-b542-7296217526cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.035383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650a3d5-98ce-4f1e-b542-7296217526cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.037407 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650a3d5-98ce-4f1e-b542-7296217526cf admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.065386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1039/2089] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:26 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:27.069671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0737a-7246-4101-a377-1923e76de3a2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.082085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0737a-7246-4101-a377-1923e76de3a2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.095119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:27.142718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06d394-de82-41cb-ae5a-2fd311e221ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.143397 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06d394-de82-41cb-ae5a-2fd311e221ec admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.143709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06d394-de82-41cb-ae5a-2fd311e221ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.143984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06d394-de82-41cb-ae5a-2fd311e221ec admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.144675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06d394-de82-41cb-ae5a-2fd311e221ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.146611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06d394-de82-41cb-ae5a-2fd311e221ec admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.167320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1051/2090] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:27 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:27.171657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650a3d5-98ce-4f1e-b542-7296217526cf admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.182283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650a3d5-98ce-4f1e-b542-7296217526cf admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.186935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88995082-627f-4fd4-909e-d6c6b8e26483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.187638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88995082-627f-4fd4-909e-d6c6b8e26483 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.187967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88995082-627f-4fd4-909e-d6c6b8e26483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.188257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88995082-627f-4fd4-909e-d6c6b8e26483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.188995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88995082-627f-4fd4-909e-d6c6b8e26483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.190607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88995082-627f-4fd4-909e-d6c6b8e26483 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.204442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1040/2091] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:27.208811 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06d394-de82-41cb-ae5a-2fd311e221ec admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.219383 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06d394-de82-41cb-ae5a-2fd311e221ec admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.224406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afd9ab-e9c4-41ec-8d4b-4e29c442ea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.225086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afd9ab-e9c4-41ec-8d4b-4e29c442ea11 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.225377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afd9ab-e9c4-41ec-8d4b-4e29c442ea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.225699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afd9ab-e9c4-41ec-8d4b-4e29c442ea11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.226574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afd9ab-e9c4-41ec-8d4b-4e29c442ea11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.228355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afd9ab-e9c4-41ec-8d4b-4e29c442ea11 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.239757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1052/2092] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:27.244542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88995082-627f-4fd4-909e-d6c6b8e26483 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.254796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88995082-627f-4fd4-909e-d6c6b8e26483 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.260235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081c30b-6efb-4c10-88fc-e7b2c4113a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.261205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081c30b-6efb-4c10-88fc-e7b2c4113a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.261556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081c30b-6efb-4c10-88fc-e7b2c4113a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.262021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081c30b-6efb-4c10-88fc-e7b2c4113a5a admin admin] PATH_INFO: `/v3/projects/40bac2f9ca0f43d8a103b4c93776bd07/users/44298a83af4d48a3a83f001bfefe2164/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.278835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081c30b-6efb-4c10-88fc-e7b2c4113a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=40bac2f9ca0f43d8a103b4c93776bd07, user_id=44298a83af4d48a3a83f001bfefe2164, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.282069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081c30b-6efb-4c10-88fc-e7b2c4113a5a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.296777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1041/2093] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:27 2022] PUT /identity/v3/projects/40bac2f9ca0f43d8a103b4c93776bd07/users/44298a83af4d48a3a83f001bfefe2164/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:27.308839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b383-f924-436b-bee7-16da693f7311 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.309131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b383-f924-436b-bee7-16da693f7311 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.309437 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b383-f924-436b-bee7-16da693f7311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.338570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bfa8b383-f924-436b-bee7-16da693f7311 None None] MFA Rules not processed for user `44298a83af4d48a3a83f001bfefe2164`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:27.339814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bfa8b383-f924-436b-bee7-16da693f7311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:27.408755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1053/2094] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:27.413455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081c30b-6efb-4c10-88fc-e7b2c4113a5a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.486641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081c30b-6efb-4c10-88fc-e7b2c4113a5a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.491720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2db7f-b299-4be9-886b-abf222c374e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.492498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2db7f-b299-4be9-886b-abf222c374e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.492880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2db7f-b299-4be9-886b-abf222c374e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.493248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2db7f-b299-4be9-886b-abf222c374e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.494032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2db7f-b299-4be9-886b-abf222c374e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.496365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2db7f-b299-4be9-886b-abf222c374e8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.526868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1042/2095] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:27 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:27.530885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8b383-f924-436b-bee7-16da693f7311 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.543551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8b383-f924-436b-bee7-16da693f7311 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.606154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df413a-6497-492c-9864-5e72d2473a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.606875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df413a-6497-492c-9864-5e72d2473a4f admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.607170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df413a-6497-492c-9864-5e72d2473a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.607521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df413a-6497-492c-9864-5e72d2473a4f admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.608218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df413a-6497-492c-9864-5e72d2473a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.610163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df413a-6497-492c-9864-5e72d2473a4f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.637473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1054/2096] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:27 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:27.641538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2db7f-b299-4be9-886b-abf222c374e8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.652394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2db7f-b299-4be9-886b-abf222c374e8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.658153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783b0e4-6dc1-4bd6-99a2-ed6b3c95df98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.658914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783b0e4-6dc1-4bd6-99a2-ed6b3c95df98 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.659245 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783b0e4-6dc1-4bd6-99a2-ed6b3c95df98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.659791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783b0e4-6dc1-4bd6-99a2-ed6b3c95df98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.660529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783b0e4-6dc1-4bd6-99a2-ed6b3c95df98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.662500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783b0e4-6dc1-4bd6-99a2-ed6b3c95df98 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.673881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1043/2097] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:27.678361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df413a-6497-492c-9864-5e72d2473a4f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.688825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df413a-6497-492c-9864-5e72d2473a4f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.694293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84648551-b174-42d5-8c58-faab393cc604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.695042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84648551-b174-42d5-8c58-faab393cc604 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.695329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84648551-b174-42d5-8c58-faab393cc604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.695643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84648551-b174-42d5-8c58-faab393cc604 admin admin] PATH_INFO: `/v3/projects/87a0369536f345b4ac6b1e4590bdc55d/users/2ddd96dcec2f45f3bc7a58fe85d0ada1/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.712838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:27.716322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84648551-b174-42d5-8c58-faab393cc604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a0369536f345b4ac6b1e4590bdc55d, user_id=2ddd96dcec2f45f3bc7a58fe85d0ada1, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.720116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84648551-b174-42d5-8c58-faab393cc604 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.739996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1055/2098] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:27 2022] PUT /identity/v3/projects/87a0369536f345b4ac6b1e4590bdc55d/users/2ddd96dcec2f45f3bc7a58fe85d0ada1/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:27.744006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783b0e4-6dc1-4bd6-99a2-ed6b3c95df98 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.755339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783b0e4-6dc1-4bd6-99a2-ed6b3c95df98 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.803953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:27.824694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56dd96-6270-4be6-b35d-2d85569f508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.825787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56dd96-6270-4be6-b35d-2d85569f508a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.825787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56dd96-6270-4be6-b35d-2d85569f508a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.826016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56dd96-6270-4be6-b35d-2d85569f508a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.826773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56dd96-6270-4be6-b35d-2d85569f508a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.828814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56dd96-6270-4be6-b35d-2d85569f508a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.849956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1044/2099] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:43:27.855071 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84648551-b174-42d5-8c58-faab393cc604 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.879561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56dd96-6270-4be6-b35d-2d85569f508a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:27.883047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84648551-b174-42d5-8c58-faab393cc604 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.885092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0e56dd96-6270-4be6-b35d-2d85569f508a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:27.893898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489ed92-cf16-45b1-a5d3-9836ab202809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.894845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489ed92-cf16-45b1-a5d3-9836ab202809 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.895134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489ed92-cf16-45b1-a5d3-9836ab202809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.895427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489ed92-cf16-45b1-a5d3-9836ab202809 admin admin] PATH_INFO: `/v3/projects/87a0369536f345b4ac6b1e4590bdc55d/users/2ddd96dcec2f45f3bc7a58fe85d0ada1/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.897525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489ed92-cf16-45b1-a5d3-9836ab202809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a0369536f345b4ac6b1e4590bdc55d, user_id=2ddd96dcec2f45f3bc7a58fe85d0ada1, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.900273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489ed92-cf16-45b1-a5d3-9836ab202809 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.903951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56dd96-6270-4be6-b35d-2d85569f508a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:27.911754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:27.912953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.913258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1056/2100] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:27 2022] PUT /identity/v3/projects/87a0369536f345b4ac6b1e4590bdc55d/users/2ddd96dcec2f45f3bc7a58fe85d0ada1/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:27.913740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.914163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] PATH_INFO: `/v3/users/dd393deb44594fbfa21630a7bbdb5c62` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:27.915579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd393deb44594fbfa21630a7bbdb5c62)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:27.917851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:27.983376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfc21d-d87f-4292-b3b7-c00668b6ce4d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:27.983707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfc21d-d87f-4292-b3b7-c00668b6ce4d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:27.983997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfc21d-d87f-4292-b3b7-c00668b6ce4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:28.002966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd393deb44594fbfa21630a7bbdb5c62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.003745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd393deb44594fbfa21630a7bbdb5c62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.013617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-54cfc21d-d87f-4292-b3b7-c00668b6ce4d None None] MFA Rules not processed for user `2ddd96dcec2f45f3bc7a58fe85d0ada1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:28.014884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-54cfc21d-d87f-4292-b3b7-c00668b6ce4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:28.018770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd393deb44594fbfa21630a7bbdb5c62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.025817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd393deb44594fbfa21630a7bbdb5c62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.026622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-528b0ef7-551a-4cd8-b6af-285fae041d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd393deb44594fbfa21630a7bbdb5c62'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.049020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1045/2101] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:27 2022] DELETE /identity/v3/users/dd393deb44594fbfa21630a7bbdb5c62 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:28.058490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7078a-c9de-48e1-a7bd-c6f5cbe1cb77 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:28.059067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7078a-c9de-48e1-a7bd-c6f5cbe1cb77 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:28.059520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7078a-c9de-48e1-a7bd-c6f5cbe1cb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:28.081677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-48b7078a-c9de-48e1-a7bd-c6f5cbe1cb77 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:28.083188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-48b7078a-c9de-48e1-a7bd-c6f5cbe1cb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:28.115028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1057/2102] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:28.126712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfc21d-d87f-4292-b3b7-c00668b6ce4d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:28.128158 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-54cfc21d-d87f-4292-b3b7-c00668b6ce4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:28.181533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1046/2103] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:28.193186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7078a-c9de-48e1-a7bd-c6f5cbe1cb77 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:28.194600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-48b7078a-c9de-48e1-a7bd-c6f5cbe1cb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:28.203244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfc21d-d87f-4292-b3b7-c00668b6ce4d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:28.259564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f3f83-7315-4216-9852-fe364709bb74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:28.260534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f3f83-7315-4216-9852-fe364709bb74 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:28.260999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f3f83-7315-4216-9852-fe364709bb74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:28.261470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f3f83-7315-4216-9852-fe364709bb74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:28.263679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bf8f3f83-7315-4216-9852-fe364709bb74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:28.266498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7078a-c9de-48e1-a7bd-c6f5cbe1cb77 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:28.271782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f466b2-b974-4aa0-947a-a05434204a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:28.272582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f466b2-b974-4aa0-947a-a05434204a86 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:28.272873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f466b2-b974-4aa0-947a-a05434204a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:28.273722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f466b2-b974-4aa0-947a-a05434204a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:28.283699 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:28.286856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f466b2-b974-4aa0-947a-a05434204a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:28.288998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f466b2-b974-4aa0-947a-a05434204a86 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:28.303945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:28.304156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1047/2104] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:43:28.341256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f3f83-7315-4216-9852-fe364709bb74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:28.343692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f3f83-7315-4216-9852-fe364709bb74 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:28.357157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1058/2105] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:43:28.609384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f466b2-b974-4aa0-947a-a05434204a86 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:28.612743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b0f466b2-b974-4aa0-947a-a05434204a86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:28.643924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f466b2-b974-4aa0-947a-a05434204a86 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:28.650101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:28.651150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:28.651495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:28.651835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] PATH_INFO: `/v3/projects/d654d9138940476c976ce7e9ffb82f3f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:28.653548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d654d9138940476c976ce7e9ffb82f3f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:28.655652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:28.708891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd654d9138940476c976ce7e9ffb82f3f', 'request_id': 'req-abcae466-2207-497e-980d-7a9436cfedae'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.715450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd654d9138940476c976ce7e9ffb82f3f', 'request_id': 'req-abcae466-2207-497e-980d-7a9436cfedae'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.720938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd654d9138940476c976ce7e9ffb82f3f', 'request_id': 'req-abcae466-2207-497e-980d-7a9436cfedae'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.739181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] The token cache is being invalidate because project d654d9138940476c976ce7e9ffb82f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:28.739775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.740887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-abcae466-2207-497e-980d-7a9436cfedae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:28.743010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1048/2106] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:28 2022] DELETE /identity/v3/projects/d654d9138940476c976ce7e9ffb82f3f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:28.750023 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e13f5-82ec-489a-bbd2-41593a06b716 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:28.750633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e13f5-82ec-489a-bbd2-41593a06b716 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:28.750633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e13f5-82ec-489a-bbd2-41593a06b716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:28.768557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bf9e13f5-82ec-489a-bbd2-41593a06b716 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:28.769759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bf9e13f5-82ec-489a-bbd2-41593a06b716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:28.831357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1059/2107] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:28.842530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576115d-f8fc-4fbb-96e2-a149c82db139 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:28.843023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576115d-f8fc-4fbb-96e2-a149c82db139 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:28.843114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576115d-f8fc-4fbb-96e2-a149c82db139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:28.863601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a576115d-f8fc-4fbb-96e2-a149c82db139 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:28.864850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a576115d-f8fc-4fbb-96e2-a149c82db139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:28.873950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1049/2108] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:28.878285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e13f5-82ec-489a-bbd2-41593a06b716 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:28.888369 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e13f5-82ec-489a-bbd2-41593a06b716 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:28.893439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbea385-3720-4995-b385-d40785259471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:28.894099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea385-3720-4995-b385-d40785259471 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:28.894386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea385-3720-4995-b385-d40785259471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:28.894666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea385-3720-4995-b385-d40785259471 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:28.895395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbea385-3720-4995-b385-d40785259471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:43:28.895753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbea385-3720-4995-b385-d40785259471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:28.897367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbea385-3720-4995-b385-d40785259471 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:28.910279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1060/2109] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:43:28.914676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576115d-f8fc-4fbb-96e2-a149c82db139 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:28.924603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576115d-f8fc-4fbb-96e2-a149c82db139 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:28.930312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5585ba3-164e-4be7-9b21-8eced6b28642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:28.931030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5585ba3-164e-4be7-9b21-8eced6b28642 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:28.931296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5585ba3-164e-4be7-9b21-8eced6b28642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:28.931641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5585ba3-164e-4be7-9b21-8eced6b28642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:28.932356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5585ba3-164e-4be7-9b21-8eced6b28642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:28.934370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5585ba3-164e-4be7-9b21-8eced6b28642 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:28.962550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1050/2110] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:28 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:28.966908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbea385-3720-4995-b385-d40785259471 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:28.977171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbea385-3720-4995-b385-d40785259471 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:29.034823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223171f-4d51-41e9-a2c6-b7a39d41b809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:29.035690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223171f-4d51-41e9-a2c6-b7a39d41b809 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:29.035979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223171f-4d51-41e9-a2c6-b7a39d41b809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:29.036258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223171f-4d51-41e9-a2c6-b7a39d41b809 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:29.036946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223171f-4d51-41e9-a2c6-b7a39d41b809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:29.038892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223171f-4d51-41e9-a2c6-b7a39d41b809 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:29.060897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1061/2111] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:28 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:29.067557 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5585ba3-164e-4be7-9b21-8eced6b28642 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:29.081271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5585ba3-164e-4be7-9b21-8eced6b28642 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:29.086356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47170842-c7f9-46f8-a1fd-98479095ae6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:29.086969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47170842-c7f9-46f8-a1fd-98479095ae6d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:29.087251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47170842-c7f9-46f8-a1fd-98479095ae6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:29.087560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47170842-c7f9-46f8-a1fd-98479095ae6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:29.088326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47170842-c7f9-46f8-a1fd-98479095ae6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:29.089936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47170842-c7f9-46f8-a1fd-98479095ae6d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:29.100695 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1051/2112] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:43:29.104810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223171f-4d51-41e9-a2c6-b7a39d41b809 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:29.115968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223171f-4d51-41e9-a2c6-b7a39d41b809 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:29.121062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6a0de-223e-4ab3-95f9-9b3ed28cdbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:29.121738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a0de-223e-4ab3-95f9-9b3ed28cdbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:29.122026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a0de-223e-4ab3-95f9-9b3ed28cdbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:29.122305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a0de-223e-4ab3-95f9-9b3ed28cdbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:29.123072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6a0de-223e-4ab3-95f9-9b3ed28cdbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:29.124739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6a0de-223e-4ab3-95f9-9b3ed28cdbe5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:29.135180 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1062/2113] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:29.139365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47170842-c7f9-46f8-a1fd-98479095ae6d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:29.149751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47170842-c7f9-46f8-a1fd-98479095ae6d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:29.154497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c707198-36a7-41b6-8ce4-25b78c3582f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:29.155384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707198-36a7-41b6-8ce4-25b78c3582f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:29.155721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707198-36a7-41b6-8ce4-25b78c3582f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:29.155997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707198-36a7-41b6-8ce4-25b78c3582f7 admin admin] PATH_INFO: `/v3/projects/f4ea28cbcfe04e1c9eef66426bbe15d6/users/826d37c02f8d4464a7ed1df3ec43058c/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:29.170992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c707198-36a7-41b6-8ce4-25b78c3582f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ea28cbcfe04e1c9eef66426bbe15d6, user_id=826d37c02f8d4464a7ed1df3ec43058c, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:29.173682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c707198-36a7-41b6-8ce4-25b78c3582f7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:29.187425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1052/2114] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:29 2022] PUT /identity/v3/projects/f4ea28cbcfe04e1c9eef66426bbe15d6/users/826d37c02f8d4464a7ed1df3ec43058c/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:29.198921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98e31c-f27e-4743-8dc2-a8384fc8fb9c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:29.199254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98e31c-f27e-4743-8dc2-a8384fc8fb9c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:29.199565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98e31c-f27e-4743-8dc2-a8384fc8fb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:29.226406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5e98e31c-f27e-4743-8dc2-a8384fc8fb9c None None] MFA Rules not processed for user `826d37c02f8d4464a7ed1df3ec43058c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:29.227655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5e98e31c-f27e-4743-8dc2-a8384fc8fb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:29.256392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:29.295226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1063/2115] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:29.306196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c707198-36a7-41b6-8ce4-25b78c3582f7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:29.308129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4c707198-36a7-41b6-8ce4-25b78c3582f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:29.396170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c707198-36a7-41b6-8ce4-25b78c3582f7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:29.403596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cd017-bcce-4e40-80d5-86bb3fb55b4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:29.404244 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cd017-bcce-4e40-80d5-86bb3fb55b4a service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:29.404520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cd017-bcce-4e40-80d5-86bb3fb55b4a service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:29.404799 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cd017-bcce-4e40-80d5-86bb3fb55b4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:29.413585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cd017-bcce-4e40-80d5-86bb3fb55b4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:29.415227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cd017-bcce-4e40-80d5-86bb3fb55b4a service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:29.420191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:29.425666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1053/2116] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:29.630958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98e31c-f27e-4743-8dc2-a8384fc8fb9c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:29.698968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98e31c-f27e-4743-8dc2-a8384fc8fb9c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:29.703680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:29.704567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:29.704849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:29.705125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] PATH_INFO: `/v3/users/826d37c02f8d4464a7ed1df3ec43058c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:29.706318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=826d37c02f8d4464a7ed1df3ec43058c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:29.708307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:29.755109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '826d37c02f8d4464a7ed1df3ec43058c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:29.755796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '826d37c02f8d4464a7ed1df3ec43058c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:29.767284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '826d37c02f8d4464a7ed1df3ec43058c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:29.774175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '826d37c02f8d4464a7ed1df3ec43058c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:29.782891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '826d37c02f8d4464a7ed1df3ec43058c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:29.792451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1064/2117] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:29 2022] DELETE /identity/v3/users/826d37c02f8d4464a7ed1df3ec43058c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:29.797781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959f7d6-a7bd-4794-a21f-eadf1a9d1ecc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:29.798133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959f7d6-a7bd-4794-a21f-eadf1a9d1ecc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:29.798462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959f7d6-a7bd-4794-a21f-eadf1a9d1ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:29.821691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b959f7d6-a7bd-4794-a21f-eadf1a9d1ecc None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:29.822953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b959f7d6-a7bd-4794-a21f-eadf1a9d1ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:29.885576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1054/2118] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:29.895195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:29.896906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:29.952776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:29.958695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0647981-5aca-4f4c-8ac1-6322219cc416 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:29.964091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f3284-c89c-498e-8944-d26c6d035df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:29.964766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f3284-c89c-498e-8944-d26c6d035df3 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:29.965062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f3284-c89c-498e-8944-d26c6d035df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:29.965371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f3284-c89c-498e-8944-d26c6d035df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:29.975592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f3284-c89c-498e-8944-d26c6d035df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:29.977048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f3284-c89c-498e-8944-d26c6d035df3 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:29.986339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1065/2119] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:30.273860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959f7d6-a7bd-4794-a21f-eadf1a9d1ecc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:30.275325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b959f7d6-a7bd-4794-a21f-eadf1a9d1ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:30.287966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959f7d6-a7bd-4794-a21f-eadf1a9d1ecc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:30.292683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:30.293594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.293880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.294158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] PATH_INFO: `/v3/projects/f4ea28cbcfe04e1c9eef66426bbe15d6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.295404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ea28cbcfe04e1c9eef66426bbe15d6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:30.297379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:30.347796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ea28cbcfe04e1c9eef66426bbe15d6', 'request_id': 'req-982f4f2e-d9f1-4e48-8cff-e16708074c50'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:30.354431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ea28cbcfe04e1c9eef66426bbe15d6', 'request_id': 'req-982f4f2e-d9f1-4e48-8cff-e16708074c50'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:30.360408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ea28cbcfe04e1c9eef66426bbe15d6', 'request_id': 'req-982f4f2e-d9f1-4e48-8cff-e16708074c50'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:30.373721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] The token cache is being invalidate because project f4ea28cbcfe04e1c9eef66426bbe15d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:30.374019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:30.374665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-982f4f2e-d9f1-4e48-8cff-e16708074c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:30.376301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1055/2120] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:30 2022] DELETE /identity/v3/projects/f4ea28cbcfe04e1c9eef66426bbe15d6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:30.381440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d1c63-ab55-4d64-ae98-0cb7655e0fd1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.381729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d1c63-ab55-4d64-ae98-0cb7655e0fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.382009 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d1c63-ab55-4d64-ae98-0cb7655e0fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.400263 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5f9d1c63-ab55-4d64-ae98-0cb7655e0fd1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:30.401397 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5f9d1c63-ab55-4d64-ae98-0cb7655e0fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:30.471124 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1066/2121] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:30.482192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5000f-8e74-4d7e-a8f1-0c76f244ebfd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.482479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5000f-8e74-4d7e-a8f1-0c76f244ebfd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.482768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5000f-8e74-4d7e-a8f1-0c76f244ebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.501627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-44c5000f-8e74-4d7e-a8f1-0c76f244ebfd None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:30.502828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-44c5000f-8e74-4d7e-a8f1-0c76f244ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:30.511305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1056/2122] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:30.515575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d1c63-ab55-4d64-ae98-0cb7655e0fd1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:30.525557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d1c63-ab55-4d64-ae98-0cb7655e0fd1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:30.530597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ece276-fd39-41f9-bde9-fe74b6a713c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:30.531245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ece276-fd39-41f9-bde9-fe74b6a713c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.531567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ece276-fd39-41f9-bde9-fe74b6a713c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.531846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ece276-fd39-41f9-bde9-fe74b6a713c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.532567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ece276-fd39-41f9-bde9-fe74b6a713c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:43:30.532868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ece276-fd39-41f9-bde9-fe74b6a713c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:30.534473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ece276-fd39-41f9-bde9-fe74b6a713c5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:30.546001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1067/2123] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:43:30.550514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5000f-8e74-4d7e-a8f1-0c76f244ebfd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:30.560038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5000f-8e74-4d7e-a8f1-0c76f244ebfd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:30.565457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c532e28-3f78-446f-8c40-7a03af48e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:30.566147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c532e28-3f78-446f-8c40-7a03af48e3bb admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.566743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c532e28-3f78-446f-8c40-7a03af48e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.567023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c532e28-3f78-446f-8c40-7a03af48e3bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.567735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c532e28-3f78-446f-8c40-7a03af48e3bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:30.570245 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c532e28-3f78-446f-8c40-7a03af48e3bb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:30.597647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1057/2124] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:30 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:30.601866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ece276-fd39-41f9-bde9-fe74b6a713c5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:30.612224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ece276-fd39-41f9-bde9-fe74b6a713c5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:30.670230 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6782aa-f9a4-4370-8a87-c54bc4601499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:30.670819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6782aa-f9a4-4370-8a87-c54bc4601499 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.671103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6782aa-f9a4-4370-8a87-c54bc4601499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.671373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6782aa-f9a4-4370-8a87-c54bc4601499 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.672091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6782aa-f9a4-4370-8a87-c54bc4601499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:30.674009 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6782aa-f9a4-4370-8a87-c54bc4601499 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:30.695246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1068/2125] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:30 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:30.699360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c532e28-3f78-446f-8c40-7a03af48e3bb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:30.709690 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c532e28-3f78-446f-8c40-7a03af48e3bb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:30.714547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3982fc-c8b2-42e4-9c86-6eebef9fac2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:30.715203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3982fc-c8b2-42e4-9c86-6eebef9fac2b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.715483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3982fc-c8b2-42e4-9c86-6eebef9fac2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.715778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3982fc-c8b2-42e4-9c86-6eebef9fac2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.716512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3982fc-c8b2-42e4-9c86-6eebef9fac2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:30.718144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3982fc-c8b2-42e4-9c86-6eebef9fac2b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:30.728346 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1058/2126] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:30.732049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6782aa-f9a4-4370-8a87-c54bc4601499 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:30.742085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6782aa-f9a4-4370-8a87-c54bc4601499 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:30.747060 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f3e37-ed46-49bd-886c-f7c3dcdf06fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:30.747755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f3e37-ed46-49bd-886c-f7c3dcdf06fe admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.748072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f3e37-ed46-49bd-886c-f7c3dcdf06fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.748370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f3e37-ed46-49bd-886c-f7c3dcdf06fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.749095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f3e37-ed46-49bd-886c-f7c3dcdf06fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:30.750737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f3e37-ed46-49bd-886c-f7c3dcdf06fe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:30.761252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1069/2127] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:30.765871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3982fc-c8b2-42e4-9c86-6eebef9fac2b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:30.776338 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3982fc-c8b2-42e4-9c86-6eebef9fac2b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:30.781734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3d368-cbfd-4e3d-98aa-4fead8a7b107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:30.782629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3d368-cbfd-4e3d-98aa-4fead8a7b107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.782909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3d368-cbfd-4e3d-98aa-4fead8a7b107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.783181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3d368-cbfd-4e3d-98aa-4fead8a7b107 admin admin] PATH_INFO: `/v3/projects/05f16eaa25fa43af82630d36e861c9ce/users/1952d3d8046c4011bb115c91e54a1023/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.798040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3d368-cbfd-4e3d-98aa-4fead8a7b107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f16eaa25fa43af82630d36e861c9ce, user_id=1952d3d8046c4011bb115c91e54a1023, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:30.800818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3d368-cbfd-4e3d-98aa-4fead8a7b107 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:30.815346 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1059/2128] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:30 2022] PUT /identity/v3/projects/05f16eaa25fa43af82630d36e861c9ce/users/1952d3d8046c4011bb115c91e54a1023/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:30.827355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7945fdc-3315-40ca-8240-b4268b6200f6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:30.827682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7945fdc-3315-40ca-8240-b4268b6200f6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:30.827960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7945fdc-3315-40ca-8240-b4268b6200f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:30.857576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d7945fdc-3315-40ca-8240-b4268b6200f6 None None] MFA Rules not processed for user `1952d3d8046c4011bb115c91e54a1023`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:30.858771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d7945fdc-3315-40ca-8240-b4268b6200f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:30.906039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:30.927499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1070/2129] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:30.932197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3d368-cbfd-4e3d-98aa-4fead8a7b107 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:30.953552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:31.003016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3d368-cbfd-4e3d-98aa-4fead8a7b107 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:31.008062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93b6d2-34a5-42f2-85e1-162850988864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:31.008726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b6d2-34a5-42f2-85e1-162850988864 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:31.009015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b6d2-34a5-42f2-85e1-162850988864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:31.009300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b6d2-34a5-42f2-85e1-162850988864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:31.010000 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93b6d2-34a5-42f2-85e1-162850988864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:31.011975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93b6d2-34a5-42f2-85e1-162850988864 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:31.040082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1060/2130] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:30 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:31.044380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7945fdc-3315-40ca-8240-b4268b6200f6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:31.055415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7945fdc-3315-40ca-8240-b4268b6200f6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:31.121923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d37134-77ff-47ee-b7a3-2c538d8c8ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:31.122602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d37134-77ff-47ee-b7a3-2c538d8c8ada admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:31.122890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d37134-77ff-47ee-b7a3-2c538d8c8ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:31.123162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d37134-77ff-47ee-b7a3-2c538d8c8ada admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:31.123886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d37134-77ff-47ee-b7a3-2c538d8c8ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:31.125844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d37134-77ff-47ee-b7a3-2c538d8c8ada admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:31.147232 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1071/2131] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:31 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:31.152103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93b6d2-34a5-42f2-85e1-162850988864 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:31.168040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93b6d2-34a5-42f2-85e1-162850988864 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:31.174314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839e836-2f2f-429d-82f6-9fb8b9a8eacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:31.174988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839e836-2f2f-429d-82f6-9fb8b9a8eacc admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:31.175276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839e836-2f2f-429d-82f6-9fb8b9a8eacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:31.175589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839e836-2f2f-429d-82f6-9fb8b9a8eacc admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:31.176363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839e836-2f2f-429d-82f6-9fb8b9a8eacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:31.178051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839e836-2f2f-429d-82f6-9fb8b9a8eacc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:31.189772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1061/2132] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:31.194195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d37134-77ff-47ee-b7a3-2c538d8c8ada admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:31.204755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d37134-77ff-47ee-b7a3-2c538d8c8ada admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:31.211119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da952e-61f0-4cee-9452-dd99eaebcca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:31.212051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da952e-61f0-4cee-9452-dd99eaebcca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:31.212332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da952e-61f0-4cee-9452-dd99eaebcca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:31.212611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da952e-61f0-4cee-9452-dd99eaebcca5 admin admin] PATH_INFO: `/v3/projects/7e6f6cc6c57948f99f6e5ba489e901e0/users/53918f6e675a481486440b8571be6e15/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:31.229694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da952e-61f0-4cee-9452-dd99eaebcca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6f6cc6c57948f99f6e5ba489e901e0, user_id=53918f6e675a481486440b8571be6e15, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:31.232457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da952e-61f0-4cee-9452-dd99eaebcca5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:31.246534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1072/2133] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:31 2022] PUT /identity/v3/projects/7e6f6cc6c57948f99f6e5ba489e901e0/users/53918f6e675a481486440b8571be6e15/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:31.250666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839e836-2f2f-429d-82f6-9fb8b9a8eacc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:31.260814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839e836-2f2f-429d-82f6-9fb8b9a8eacc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:31.320804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e4cc2-61ea-40cf-ae23-a5932df8713f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:31.321489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e4cc2-61ea-40cf-ae23-a5932df8713f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:31.321771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e4cc2-61ea-40cf-ae23-a5932df8713f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:31.322072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e4cc2-61ea-40cf-ae23-a5932df8713f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:31.322804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e4cc2-61ea-40cf-ae23-a5932df8713f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:31.324509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e4cc2-61ea-40cf-ae23-a5932df8713f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:31.335559 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1062/2134] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:31.339693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da952e-61f0-4cee-9452-dd99eaebcca5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:31.350598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da952e-61f0-4cee-9452-dd99eaebcca5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:31.355578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7ef77-7466-49fa-9097-93be829f7e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:31.356532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7ef77-7466-49fa-9097-93be829f7e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:31.356887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7ef77-7466-49fa-9097-93be829f7e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:31.357170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7ef77-7466-49fa-9097-93be829f7e4d admin admin] PATH_INFO: `/v3/projects/7e6f6cc6c57948f99f6e5ba489e901e0/users/53918f6e675a481486440b8571be6e15/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:31.359260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7ef77-7466-49fa-9097-93be829f7e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6f6cc6c57948f99f6e5ba489e901e0, user_id=53918f6e675a481486440b8571be6e15, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:31.362119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7ef77-7466-49fa-9097-93be829f7e4d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:31.373115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1073/2135] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:31 2022] PUT /identity/v3/projects/7e6f6cc6c57948f99f6e5ba489e901e0/users/53918f6e675a481486440b8571be6e15/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:31.384941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68799e1d-9a5e-42dd-9297-52b04f499889 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:31.384941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68799e1d-9a5e-42dd-9297-52b04f499889 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:31.384941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68799e1d-9a5e-42dd-9297-52b04f499889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:31.414083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-68799e1d-9a5e-42dd-9297-52b04f499889 None None] MFA Rules not processed for user `53918f6e675a481486440b8571be6e15`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:31.415293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-68799e1d-9a5e-42dd-9297-52b04f499889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:31.494175 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1063/2136] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:31.505074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7ef77-7466-49fa-9097-93be829f7e4d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:31.507079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a6e7ef77-7466-49fa-9097-93be829f7e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:31.564368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7ef77-7466-49fa-9097-93be829f7e4d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:31.568725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3b066-dc44-4643-9698-4fef4be992c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:31.569368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3b066-dc44-4643-9698-4fef4be992c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:31.569649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3b066-dc44-4643-9698-4fef4be992c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:31.569923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3b066-dc44-4643-9698-4fef4be992c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:31.579200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3b066-dc44-4643-9698-4fef4be992c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:31.580674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3b066-dc44-4643-9698-4fef4be992c0 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:31.589806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1074/2137] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:32.190623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68799e1d-9a5e-42dd-9297-52b04f499889 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:32.209133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:32.259325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68799e1d-9a5e-42dd-9297-52b04f499889 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:32.263959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:32.264837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:32.265116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:32.265390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] PATH_INFO: `/v3/users/1952d3d8046c4011bb115c91e54a1023` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:32.266605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1952d3d8046c4011bb115c91e54a1023)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:32.268581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:32.327396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1952d3d8046c4011bb115c91e54a1023'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.328189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1952d3d8046c4011bb115c91e54a1023'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.341460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1952d3d8046c4011bb115c91e54a1023'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.348455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1952d3d8046c4011bb115c91e54a1023'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.349215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1952d3d8046c4011bb115c91e54a1023'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.362635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1064/2138] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:32 2022] DELETE /identity/v3/users/1952d3d8046c4011bb115c91e54a1023 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:32.367295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9bab-27b5-441a-9950-9d0c23c67a96 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:32.367806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9bab-27b5-441a-9950-9d0c23c67a96 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:32.367921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9bab-27b5-441a-9950-9d0c23c67a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:32.386780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3fbe9bab-27b5-441a-9950-9d0c23c67a96 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:32.388111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3fbe9bab-27b5-441a-9950-9d0c23c67a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:32.428360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:32.452756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1075/2139] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:32.462565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:32.464355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:32.523149 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da7418-fef5-4b4d-b4c8-a6ddf5496eed admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:32.526822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8ce86-16d1-48c1-b0b7-94d1d585f8cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:32.527494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8ce86-16d1-48c1-b0b7-94d1d585f8cb service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:32.527803 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8ce86-16d1-48c1-b0b7-94d1d585f8cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:32.528049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8ce86-16d1-48c1-b0b7-94d1d585f8cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:32.538539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8ce86-16d1-48c1-b0b7-94d1d585f8cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:32.540037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8ce86-16d1-48c1-b0b7-94d1d585f8cb service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:32.550169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1065/2140] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:32.821968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9bab-27b5-441a-9950-9d0c23c67a96 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:32.823557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3fbe9bab-27b5-441a-9950-9d0c23c67a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:32.836914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9bab-27b5-441a-9950-9d0c23c67a96 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:32.843074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:32.843613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:32.843895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:32.844186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] PATH_INFO: `/v3/projects/05f16eaa25fa43af82630d36e861c9ce` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:32.845392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f16eaa25fa43af82630d36e861c9ce)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:32.847527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:32.903660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f16eaa25fa43af82630d36e861c9ce', 'request_id': 'req-586dfc46-075e-4057-baac-b1d124d1f68b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.912589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f16eaa25fa43af82630d36e861c9ce', 'request_id': 'req-586dfc46-075e-4057-baac-b1d124d1f68b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.917900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f16eaa25fa43af82630d36e861c9ce', 'request_id': 'req-586dfc46-075e-4057-baac-b1d124d1f68b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.928594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] The token cache is being invalidate because project 05f16eaa25fa43af82630d36e861c9ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:32.928884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.929501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:32.931077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1076/2141] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:32 2022] DELETE /identity/v3/projects/05f16eaa25fa43af82630d36e861c9ce => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:32.935057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8ce86-16d1-48c1-b0b7-94d1d585f8cb service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:32.945096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8ce86-16d1-48c1-b0b7-94d1d585f8cb service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:32.978399 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:33.005189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:33.006082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.006364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.006638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] PATH_INFO: `/v3/users/53918f6e675a481486440b8571be6e15` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:33.007871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53918f6e675a481486440b8571be6e15)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:33.009811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:33.065461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53918f6e675a481486440b8571be6e15'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.066116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53918f6e675a481486440b8571be6e15'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.079378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53918f6e675a481486440b8571be6e15'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.086521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53918f6e675a481486440b8571be6e15'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.087220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8b0d4145-e74e-4743-ac69-9fcbd334dc50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53918f6e675a481486440b8571be6e15'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.101636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1066/2142] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:32 2022] DELETE /identity/v3/users/53918f6e675a481486440b8571be6e15 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:43:33.384897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:33.386447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:33.404310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586dfc46-075e-4057-baac-b1d124d1f68b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:33.467559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:33.468476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.468755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.469026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] PATH_INFO: `/v3/projects/7e6f6cc6c57948f99f6e5ba489e901e0` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:33.470218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6f6cc6c57948f99f6e5ba489e901e0)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:33.472184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:33.518382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6f6cc6c57948f99f6e5ba489e901e0', 'request_id': 'req-8e0cca8c-967a-490e-8781-f162010a825e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.528082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6f6cc6c57948f99f6e5ba489e901e0', 'request_id': 'req-8e0cca8c-967a-490e-8781-f162010a825e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.536751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6f6cc6c57948f99f6e5ba489e901e0', 'request_id': 'req-8e0cca8c-967a-490e-8781-f162010a825e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.539796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:33.555922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] The token cache is being invalidate because project 7e6f6cc6c57948f99f6e5ba489e901e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:33.556334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.557473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e0cca8c-967a-490e-8781-f162010a825e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:33.560378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1077/2143] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:33 2022] DELETE /identity/v3/projects/7e6f6cc6c57948f99f6e5ba489e901e0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:33.566706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df6644-8f89-4cdf-88ce-1e13ac1c7590 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.567178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df6644-8f89-4cdf-88ce-1e13ac1c7590 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.567276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df6644-8f89-4cdf-88ce-1e13ac1c7590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:33.585917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-23df6644-8f89-4cdf-88ce-1e13ac1c7590 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:33.587287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-23df6644-8f89-4cdf-88ce-1e13ac1c7590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:33.654319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1067/2144] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:33.662051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152d50f-41b7-4143-b048-a4d40c8351dc None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.662402 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152d50f-41b7-4143-b048-a4d40c8351dc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.662717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152d50f-41b7-4143-b048-a4d40c8351dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:33.683363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1152d50f-41b7-4143-b048-a4d40c8351dc None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:33.684607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1152d50f-41b7-4143-b048-a4d40c8351dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:33.694344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1078/2145] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:33.698450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df6644-8f89-4cdf-88ce-1e13ac1c7590 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:33.710373 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df6644-8f89-4cdf-88ce-1e13ac1c7590 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:33.715361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0a07d-d085-4a33-a723-5b7246c3a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:33.716533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0a07d-d085-4a33-a723-5b7246c3a944 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.717583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0a07d-d085-4a33-a723-5b7246c3a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.718201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0a07d-d085-4a33-a723-5b7246c3a944 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:33.719143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0a07d-d085-4a33-a723-5b7246c3a944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:43:33.719665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0a07d-d085-4a33-a723-5b7246c3a944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:33.721827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0a07d-d085-4a33-a723-5b7246c3a944 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:33.743634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1068/2146] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:43:33.745328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152d50f-41b7-4143-b048-a4d40c8351dc None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:33.755845 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152d50f-41b7-4143-b048-a4d40c8351dc None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:33.760840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f1daa-bc6c-4637-8d66-0761dbf8f43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:33.761510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f1daa-bc6c-4637-8d66-0761dbf8f43d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.761921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f1daa-bc6c-4637-8d66-0761dbf8f43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.762471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f1daa-bc6c-4637-8d66-0761dbf8f43d admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:33.763295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f1daa-bc6c-4637-8d66-0761dbf8f43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:33.765134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f1daa-bc6c-4637-8d66-0761dbf8f43d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:33.795641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1079/2147] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:33 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:43:33.799757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0a07d-d085-4a33-a723-5b7246c3a944 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:33.810457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0a07d-d085-4a33-a723-5b7246c3a944 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:33.851788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:33.869887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a48dc-d12e-4168-9df0-443af245198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:33.870659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a48dc-d12e-4168-9df0-443af245198a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.870999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a48dc-d12e-4168-9df0-443af245198a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.871366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a48dc-d12e-4168-9df0-443af245198a admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:33.872109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a48dc-d12e-4168-9df0-443af245198a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:33.874067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a48dc-d12e-4168-9df0-443af245198a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:33.894919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1069/2148] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:33 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:33.899226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f1daa-bc6c-4637-8d66-0761dbf8f43d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:33.909399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f1daa-bc6c-4637-8d66-0761dbf8f43d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:33.915087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0dce9-4c34-4990-999e-7471ffdf880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:33.915805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0dce9-4c34-4990-999e-7471ffdf880f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.916090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0dce9-4c34-4990-999e-7471ffdf880f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.916362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0dce9-4c34-4990-999e-7471ffdf880f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:33.917085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0dce9-4c34-4990-999e-7471ffdf880f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:33.918725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0dce9-4c34-4990-999e-7471ffdf880f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:33.929984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1080/2149] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:43:33.934204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a48dc-d12e-4168-9df0-443af245198a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:33.945252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a48dc-d12e-4168-9df0-443af245198a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:33.950491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a15faa-aa8c-468d-817a-f7b5c76d06f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:33.951478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a15faa-aa8c-468d-817a-f7b5c76d06f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.951788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a15faa-aa8c-468d-817a-f7b5c76d06f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.952136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a15faa-aa8c-468d-817a-f7b5c76d06f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:33.952872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a15faa-aa8c-468d-817a-f7b5c76d06f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:33.954500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a15faa-aa8c-468d-817a-f7b5c76d06f7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:33.968369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1070/2150] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:33.972771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0dce9-4c34-4990-999e-7471ffdf880f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:33.983432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0dce9-4c34-4990-999e-7471ffdf880f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:33.989124 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b6694-5c65-494d-98d8-520e8693f1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:33.990027 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b6694-5c65-494d-98d8-520e8693f1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:33.990367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b6694-5c65-494d-98d8-520e8693f1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:33.990656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b6694-5c65-494d-98d8-520e8693f1b1 admin admin] PATH_INFO: `/v3/projects/054bd78b264946b59cfe5911daf1868e/users/eb3019f8ecf841c4a91f7af9295fb5de/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:34.006371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b6694-5c65-494d-98d8-520e8693f1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=054bd78b264946b59cfe5911daf1868e, user_id=eb3019f8ecf841c4a91f7af9295fb5de, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:34.009125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b6694-5c65-494d-98d8-520e8693f1b1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:34.022698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1081/2151] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:33 2022] PUT /identity/v3/projects/054bd78b264946b59cfe5911daf1868e/users/eb3019f8ecf841c4a91f7af9295fb5de/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:34.034568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8d2a6-0111-4e35-b8e5-c6dbe662134e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:34.034862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8d2a6-0111-4e35-b8e5-c6dbe662134e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:34.035142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8d2a6-0111-4e35-b8e5-c6dbe662134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:34.065654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-83d8d2a6-0111-4e35-b8e5-c6dbe662134e None None] MFA Rules not processed for user `eb3019f8ecf841c4a91f7af9295fb5de`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:34.067611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-83d8d2a6-0111-4e35-b8e5-c6dbe662134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:34.104533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b6694-5c65-494d-98d8-520e8693f1b1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:34.106979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f16b6694-5c65-494d-98d8-520e8693f1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:34.138083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1071/2152] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:34.150114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8d2a6-0111-4e35-b8e5-c6dbe662134e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:34.151675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-83d8d2a6-0111-4e35-b8e5-c6dbe662134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:34.167870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b6694-5c65-494d-98d8-520e8693f1b1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:34.172989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:34.173897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:34.174392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:34.174680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:34.176513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:34.211292 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8d2a6-0111-4e35-b8e5-c6dbe662134e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:34.214945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867120a-2e04-4653-abae-fc4c310452ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc21faf2237d405bba03e5309d93629f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:34.215645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867120a-2e04-4653-abae-fc4c310452ef service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:34.215929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867120a-2e04-4653-abae-fc4c310452ef service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:34.216208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867120a-2e04-4653-abae-fc4c310452ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:34.227955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867120a-2e04-4653-abae-fc4c310452ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:34.229403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867120a-2e04-4653-abae-fc4c310452ef service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:34.236954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:34.237906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:34.239312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1072/2153] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:43:34.246751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1082/2154] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:43:34.830480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867120a-2e04-4653-abae-fc4c310452ef service glance] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:34.831999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7867120a-2e04-4653-abae-fc4c310452ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:34.855164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867120a-2e04-4653-abae-fc4c310452ef service glance] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:34.862041 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19218eba-20a8-4e32-9893-444f25fb9b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:34.863297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19218eba-20a8-4e32-9893-444f25fb9b71 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:34.863830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19218eba-20a8-4e32-9893-444f25fb9b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:34.864298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19218eba-20a8-4e32-9893-444f25fb9b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:34.867533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-19218eba-20a8-4e32-9893-444f25fb9b71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:34.956814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:34.973505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19218eba-20a8-4e32-9893-444f25fb9b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:34.974955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19218eba-20a8-4e32-9893-444f25fb9b71 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:34.985192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1073/2155] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:38.351676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:38.353349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:38.407300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:38.425717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133660ae-b7b0-448a-ac3f-bc96a184f1d0 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:38.431377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:38.432525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:38.433264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:38.433264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] PATH_INFO: `/v3/users/85c2b9f4ee6a4419980c1649de7c6a55` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:38.434414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c2b9f4ee6a4419980c1649de7c6a55)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:38.436447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:38.496138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c2b9f4ee6a4419980c1649de7c6a55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.496819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c2b9f4ee6a4419980c1649de7c6a55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.509325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c2b9f4ee6a4419980c1649de7c6a55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.516600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c2b9f4ee6a4419980c1649de7c6a55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.525291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c2b9f4ee6a4419980c1649de7c6a55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.531666 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1083/2156] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:38 2022] DELETE /identity/v3/users/85c2b9f4ee6a4419980c1649de7c6a55 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:38.536397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f10c-53d7-4748-a2ad-da1600b36595 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:38.536938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f10c-53d7-4748-a2ad-da1600b36595 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:38.537042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f10c-53d7-4748-a2ad-da1600b36595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:38.556092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-10f5f10c-53d7-4748-a2ad-da1600b36595 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:38.557281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-10f5f10c-53d7-4748-a2ad-da1600b36595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:38.622658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1074/2157] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:38.633152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:38.635297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:38.696296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b80f1-1155-4ef2-a9fc-9195293ddc60 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:38.700711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ca229-28ed-4772-9bcc-fc28b2822a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:38.701406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ca229-28ed-4772-9bcc-fc28b2822a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:38.701694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ca229-28ed-4772-9bcc-fc28b2822a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:38.702179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ca229-28ed-4772-9bcc-fc28b2822a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:38.713225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ca229-28ed-4772-9bcc-fc28b2822a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:38.714726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ca229-28ed-4772-9bcc-fc28b2822a60 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:38.724684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1084/2158] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:38.883387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5f10c-53d7-4748-a2ad-da1600b36595 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:38.885379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-10f5f10c-53d7-4748-a2ad-da1600b36595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:38.897411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5f10c-53d7-4748-a2ad-da1600b36595 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:38.901958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:38.902839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:38.903124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:38.903400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] PATH_INFO: `/v3/users/bdcd1fa6d703402283a1ae45107970a7` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:38.904673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdcd1fa6d703402283a1ae45107970a7)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:38.906628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:38.963915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdcd1fa6d703402283a1ae45107970a7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.964645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdcd1fa6d703402283a1ae45107970a7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.979341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdcd1fa6d703402283a1ae45107970a7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.987236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdcd1fa6d703402283a1ae45107970a7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.988169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-564401ac-fa75-4ff0-8bbd-f8d9049c66d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdcd1fa6d703402283a1ae45107970a7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:38.999360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ca229-28ed-4772-9bcc-fc28b2822a60 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.001644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-483ca229-28ed-4772-9bcc-fc28b2822a60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.004461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1075/2159] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:38 2022] DELETE /identity/v3/users/bdcd1fa6d703402283a1ae45107970a7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:39.008768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38cf53-668b-4ccd-b34b-188607583585 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.008994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38cf53-668b-4ccd-b34b-188607583585 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.009276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38cf53-668b-4ccd-b34b-188607583585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.015864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ca229-28ed-4772-9bcc-fc28b2822a60 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.035741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3c38cf53-668b-4ccd-b34b-188607583585 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:39.036962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3c38cf53-668b-4ccd-b34b-188607583585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.112645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.112645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.112645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.112645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] PATH_INFO: `/v3/projects/4f631df7309b47e184f487903e91b38d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.112993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f631df7309b47e184f487903e91b38d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.112993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.117150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1076/2160] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:39.123793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:39.127222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38cf53-668b-4ccd-b34b-188607583585 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.128672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3c38cf53-668b-4ccd-b34b-188607583585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.134530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:39.169746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f631df7309b47e184f487903e91b38d', 'request_id': 'req-aa59969f-994f-4e6a-9b1b-70e165cf5062'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.177912 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f631df7309b47e184f487903e91b38d', 'request_id': 'req-aa59969f-994f-4e6a-9b1b-70e165cf5062'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.184324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f631df7309b47e184f487903e91b38d', 'request_id': 'req-aa59969f-994f-4e6a-9b1b-70e165cf5062'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.193372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38cf53-668b-4ccd-b34b-188607583585 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.195813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] The token cache is being invalidate because project 4f631df7309b47e184f487903e91b38d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:39.196108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.196747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-aa59969f-994f-4e6a-9b1b-70e165cf5062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.198298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1085/2161] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:38 2022] DELETE /identity/v3/projects/4f631df7309b47e184f487903e91b38d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:39.204141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba555174-28c9-40e2-8945-3e0e83b76342 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.204428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba555174-28c9-40e2-8945-3e0e83b76342 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.204707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba555174-28c9-40e2-8945-3e0e83b76342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.222584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ba555174-28c9-40e2-8945-3e0e83b76342 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:39.223831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ba555174-28c9-40e2-8945-3e0e83b76342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.242302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfbb9e-ff35-4ad7-85e0-5c37d79037b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.243070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfbb9e-ff35-4ad7-85e0-5c37d79037b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.243362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfbb9e-ff35-4ad7-85e0-5c37d79037b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.243745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfbb9e-ff35-4ad7-85e0-5c37d79037b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.245705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5fdfbb9e-ff35-4ad7-85e0-5c37d79037b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.289729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1086/2162] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:39.297641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52502e4-7b32-45a8-8b54-1bf335936a6b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.297938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52502e4-7b32-45a8-8b54-1bf335936a6b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.298212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52502e4-7b32-45a8-8b54-1bf335936a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.316923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d52502e4-7b32-45a8-8b54-1bf335936a6b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:39.317165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfbb9e-ff35-4ad7-85e0-5c37d79037b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.318097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d52502e4-7b32-45a8-8b54-1bf335936a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.318616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfbb9e-ff35-4ad7-85e0-5c37d79037b2 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.325933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1087/2163] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:39.328420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1077/2164] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:39.330131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52502e4-7b32-45a8-8b54-1bf335936a6b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.341195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52502e4-7b32-45a8-8b54-1bf335936a6b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.350822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccda52-75fb-4b4b-829d-d40ed0ee1ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.351648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccda52-75fb-4b4b-829d-d40ed0ee1ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.351937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccda52-75fb-4b4b-829d-d40ed0ee1ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.352515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccda52-75fb-4b4b-829d-d40ed0ee1ff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.353100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccda52-75fb-4b4b-829d-d40ed0ee1ff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:43:39.353414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccda52-75fb-4b4b-829d-d40ed0ee1ff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.355074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccda52-75fb-4b4b-829d-d40ed0ee1ff8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.367280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1088/2165] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:43:39.371413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfbb9e-ff35-4ad7-85e0-5c37d79037b2 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.381408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfbb9e-ff35-4ad7-85e0-5c37d79037b2 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.386541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b6b4c-bd58-44d5-8a24-0984f154d68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.387213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b6b4c-bd58-44d5-8a24-0984f154d68c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.387577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b6b4c-bd58-44d5-8a24-0984f154d68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.387778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b6b4c-bd58-44d5-8a24-0984f154d68c admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.388597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b6b4c-bd58-44d5-8a24-0984f154d68c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.390685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b6b4c-bd58-44d5-8a24-0984f154d68c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.418661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1078/2166] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:39 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:39.423104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccda52-75fb-4b4b-829d-d40ed0ee1ff8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.440893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccda52-75fb-4b4b-829d-d40ed0ee1ff8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.504912 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13077-e3a0-4dfd-a663-14005fb88031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.505612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13077-e3a0-4dfd-a663-14005fb88031 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.505904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13077-e3a0-4dfd-a663-14005fb88031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.506193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13077-e3a0-4dfd-a663-14005fb88031 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.506906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13077-e3a0-4dfd-a663-14005fb88031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.508916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13077-e3a0-4dfd-a663-14005fb88031 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.529557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1089/2167] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:39 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:39.534217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b6b4c-bd58-44d5-8a24-0984f154d68c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.544843 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b6b4c-bd58-44d5-8a24-0984f154d68c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.549969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd6937-3868-4ae2-a07a-0c8ef73508bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.550712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd6937-3868-4ae2-a07a-0c8ef73508bc admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.551039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd6937-3868-4ae2-a07a-0c8ef73508bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.551352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd6937-3868-4ae2-a07a-0c8ef73508bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.552138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd6937-3868-4ae2-a07a-0c8ef73508bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.554272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd6937-3868-4ae2-a07a-0c8ef73508bc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.567184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1079/2168] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:39.571370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13077-e3a0-4dfd-a663-14005fb88031 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.581865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13077-e3a0-4dfd-a663-14005fb88031 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.586741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7983596-8997-4cb9-b283-489849126e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.587442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7983596-8997-4cb9-b283-489849126e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.587759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7983596-8997-4cb9-b283-489849126e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.588041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7983596-8997-4cb9-b283-489849126e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.588828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7983596-8997-4cb9-b283-489849126e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.590479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7983596-8997-4cb9-b283-489849126e14 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.592986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:39.609496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1090/2169] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:39.610828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd6937-3868-4ae2-a07a-0c8ef73508bc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.623105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7983596-8997-4cb9-b283-489849126e14 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.623501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a7983596-8997-4cb9-b283-489849126e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.627218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd6937-3868-4ae2-a07a-0c8ef73508bc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.634233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c73c8-2435-41d6-8710-cf3b0122adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.635160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c73c8-2435-41d6-8710-cf3b0122adb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.635582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c73c8-2435-41d6-8710-cf3b0122adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.635771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c73c8-2435-41d6-8710-cf3b0122adb6 admin admin] PATH_INFO: `/v3/projects/c854ecea0b6943b2857d531dfbed04ad/users/0d648fae81514f87b2c0dd7f494aee46/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.643819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7983596-8997-4cb9-b283-489849126e14 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.648717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.649597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.649948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.650152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] PATH_INFO: `/v3/projects/a0edf440dd4e413da0db9a65d2c3a592` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.651549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0edf440dd4e413da0db9a65d2c3a592)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.653472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.669282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c73c8-2435-41d6-8710-cf3b0122adb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c854ecea0b6943b2857d531dfbed04ad, user_id=0d648fae81514f87b2c0dd7f494aee46, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.673171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c73c8-2435-41d6-8710-cf3b0122adb6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.690741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1080/2170] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:39 2022] PUT /identity/v3/projects/c854ecea0b6943b2857d531dfbed04ad/users/0d648fae81514f87b2c0dd7f494aee46/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:43:39.696353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08cd21-e985-405c-8c1e-a05cc6247277 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.696707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08cd21-e985-405c-8c1e-a05cc6247277 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.696996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08cd21-e985-405c-8c1e-a05cc6247277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.718267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1b08cd21-e985-405c-8c1e-a05cc6247277 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:39.719660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1b08cd21-e985-405c-8c1e-a05cc6247277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.723019 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0edf440dd4e413da0db9a65d2c3a592', 'request_id': 'req-3727c95d-340c-4dc8-bebb-9839c88bb87c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.725662 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:39.733219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0edf440dd4e413da0db9a65d2c3a592', 'request_id': 'req-3727c95d-340c-4dc8-bebb-9839c88bb87c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.739551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0edf440dd4e413da0db9a65d2c3a592', 'request_id': 'req-3727c95d-340c-4dc8-bebb-9839c88bb87c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.750943 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] The token cache is being invalidate because project a0edf440dd4e413da0db9a65d2c3a592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:39.751240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.751946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.753584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1091/2171] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:39 2022] DELETE /identity/v3/projects/a0edf440dd4e413da0db9a65d2c3a592 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:43:39.757440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.759320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.799351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1081/2172] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:39.810103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08cd21-e985-405c-8c1e-a05cc6247277 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:39.811486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1b08cd21-e985-405c-8c1e-a05cc6247277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:39.826961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727c95d-340c-4dc8-bebb-9839c88bb87c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.831625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.832512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.832796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.833075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] PATH_INFO: `/v3/users/44298a83af4d48a3a83f001bfefe2164` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.834278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44298a83af4d48a3a83f001bfefe2164)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.836342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.868062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08cd21-e985-405c-8c1e-a05cc6247277 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:39.872552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1b838-148e-4dbc-8d15-3b06108f7b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:39.873414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1b838-148e-4dbc-8d15-3b06108f7b99 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:39.873695 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1b838-148e-4dbc-8d15-3b06108f7b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:39.874001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1b838-148e-4dbc-8d15-3b06108f7b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:39.884298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1b838-148e-4dbc-8d15-3b06108f7b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:39.885796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1b838-148e-4dbc-8d15-3b06108f7b99 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:39.889298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44298a83af4d48a3a83f001bfefe2164'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.890186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44298a83af4d48a3a83f001bfefe2164'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.896530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1082/2173] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:43:39.908107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44298a83af4d48a3a83f001bfefe2164'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.914868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44298a83af4d48a3a83f001bfefe2164'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.923172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44298a83af4d48a3a83f001bfefe2164'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:39.925602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:39.930364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1092/2174] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:39 2022] DELETE /identity/v3/users/44298a83af4d48a3a83f001bfefe2164 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:40.292070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1b838-148e-4dbc-8d15-3b06108f7b99 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:40.303360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1b838-148e-4dbc-8d15-3b06108f7b99 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:40.370658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:40.371593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:40.371884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:40.372254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] PATH_INFO: `/v3/projects/40bac2f9ca0f43d8a103b4c93776bd07` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:40.373689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40bac2f9ca0f43d8a103b4c93776bd07)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:40.375929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:40.424973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40bac2f9ca0f43d8a103b4c93776bd07', 'request_id': 'req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.432224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40bac2f9ca0f43d8a103b4c93776bd07', 'request_id': 'req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.434278 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:40.438993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40bac2f9ca0f43d8a103b4c93776bd07', 'request_id': 'req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.452077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] The token cache is being invalidate because project 40bac2f9ca0f43d8a103b4c93776bd07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:40.452542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.453173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.454851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1083/2175] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:40 2022] DELETE /identity/v3/projects/40bac2f9ca0f43d8a103b4c93776bd07 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:40.459946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:40.462128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:40.478092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ffb4d-73be-420f-b5b2-1374617817c0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:40.548148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:40.551248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:40.551248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:40.551248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] PATH_INFO: `/v3/users/2ddd96dcec2f45f3bc7a58fe85d0ada1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:40.551248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ddd96dcec2f45f3bc7a58fe85d0ada1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:40.552950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:40.622577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ddd96dcec2f45f3bc7a58fe85d0ada1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.623492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ddd96dcec2f45f3bc7a58fe85d0ada1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.638705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ddd96dcec2f45f3bc7a58fe85d0ada1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.649344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ddd96dcec2f45f3bc7a58fe85d0ada1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.660079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb55573a-af5e-4642-b20c-611ac540414a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ddd96dcec2f45f3bc7a58fe85d0ada1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:40.671006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1093/2176] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:40 2022] DELETE /identity/v3/users/2ddd96dcec2f45f3bc7a58fe85d0ada1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:43:41.003701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:41.006587 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:41.031689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62518-d553-44ca-8a04-ff9bdc6c2cf8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:41.135399 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:41.136327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.136612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.136885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] PATH_INFO: `/v3/projects/87a0369536f345b4ac6b1e4590bdc55d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.138377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a0369536f345b4ac6b1e4590bdc55d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:41.140395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:41.236023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a0369536f345b4ac6b1e4590bdc55d', 'request_id': 'req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:41.247192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a0369536f345b4ac6b1e4590bdc55d', 'request_id': 'req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:41.267848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a0369536f345b4ac6b1e4590bdc55d', 'request_id': 'req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:41.286287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] The token cache is being invalidate because project 87a0369536f345b4ac6b1e4590bdc55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:41.286655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:41.287322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8bb9e2a-8761-4b68-bb25-69bcfc2acaa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:41.289412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1084/2177] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:41 2022] DELETE /identity/v3/projects/87a0369536f345b4ac6b1e4590bdc55d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:41.294828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f078-90e0-416b-8920-119d51679a2b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.294828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f078-90e0-416b-8920-119d51679a2b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.294828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f078-90e0-416b-8920-119d51679a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.315413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0999f078-90e0-416b-8920-119d51679a2b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:41.317225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0999f078-90e0-416b-8920-119d51679a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:41.383087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1094/2178] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:41.392005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d8da3-123c-4ca4-bd7b-ed1567a7e9ae None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.392299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d8da3-123c-4ca4-bd7b-ed1567a7e9ae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.392579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d8da3-123c-4ca4-bd7b-ed1567a7e9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.412728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9c5d8da3-123c-4ca4-bd7b-ed1567a7e9ae None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:41.414581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9c5d8da3-123c-4ca4-bd7b-ed1567a7e9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:41.424566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1085/2179] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:41.429118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999f078-90e0-416b-8920-119d51679a2b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:41.444481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999f078-90e0-416b-8920-119d51679a2b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:41.459682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c0fe9-b8ca-4c12-8ddc-7d41efb3805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:41.460376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c0fe9-b8ca-4c12-8ddc-7d41efb3805e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.460907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c0fe9-b8ca-4c12-8ddc-7d41efb3805e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.460907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c0fe9-b8ca-4c12-8ddc-7d41efb3805e admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.461655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c0fe9-b8ca-4c12-8ddc-7d41efb3805e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:43:41.462600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c0fe9-b8ca-4c12-8ddc-7d41efb3805e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:41.466398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c0fe9-b8ca-4c12-8ddc-7d41efb3805e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:41.489872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1095/2180] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:43:41.490627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d8da3-123c-4ca4-bd7b-ed1567a7e9ae None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:41.509418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d8da3-123c-4ca4-bd7b-ed1567a7e9ae None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:41.518156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a850ee-07fe-484d-9500-baa729264c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:41.518871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a850ee-07fe-484d-9500-baa729264c2c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.519159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a850ee-07fe-484d-9500-baa729264c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.519448 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a850ee-07fe-484d-9500-baa729264c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.520369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a850ee-07fe-484d-9500-baa729264c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:41.522630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a850ee-07fe-484d-9500-baa729264c2c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:41.554795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1086/2181] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:41 2022] POST /identity/v3/projects => generated 399 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:43:41.555975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c0fe9-b8ca-4c12-8ddc-7d41efb3805e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:41.569887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c0fe9-b8ca-4c12-8ddc-7d41efb3805e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:41.574592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:41.650946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d312e5a-c037-4402-8012-a8691d5416a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:41.652331 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d312e5a-c037-4402-8012-a8691d5416a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.652624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d312e5a-c037-4402-8012-a8691d5416a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.652909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d312e5a-c037-4402-8012-a8691d5416a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.653610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d312e5a-c037-4402-8012-a8691d5416a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:41.655697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d312e5a-c037-4402-8012-a8691d5416a5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:41.680735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1096/2182] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:41 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:43:41.681845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a850ee-07fe-484d-9500-baa729264c2c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:41.692690 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a850ee-07fe-484d-9500-baa729264c2c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:41.697414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12544686-3a3b-45a1-9d7e-d39c34990eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:41.698129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12544686-3a3b-45a1-9d7e-d39c34990eab admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.698368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12544686-3a3b-45a1-9d7e-d39c34990eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.698648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12544686-3a3b-45a1-9d7e-d39c34990eab admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.699387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12544686-3a3b-45a1-9d7e-d39c34990eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:41.701328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12544686-3a3b-45a1-9d7e-d39c34990eab admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:41.720594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1087/2183] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:43:41.721927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d312e5a-c037-4402-8012-a8691d5416a5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:41.735024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d312e5a-c037-4402-8012-a8691d5416a5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:41.739955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2e5e3-6aaa-425e-a446-8d290bedbe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:41.740649 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2e5e3-6aaa-425e-a446-8d290bedbe21 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.740937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2e5e3-6aaa-425e-a446-8d290bedbe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.741216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2e5e3-6aaa-425e-a446-8d290bedbe21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.741955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2e5e3-6aaa-425e-a446-8d290bedbe21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:41.743636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2e5e3-6aaa-425e-a446-8d290bedbe21 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:41.757839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1097/2184] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:41.759047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12544686-3a3b-45a1-9d7e-d39c34990eab admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:41.769446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12544686-3a3b-45a1-9d7e-d39c34990eab admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:41.774301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d5e3e-6b0e-4b5f-be5d-7a7304f8f3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:41.775282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d5e3e-6b0e-4b5f-be5d-7a7304f8f3e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.775809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d5e3e-6b0e-4b5f-be5d-7a7304f8f3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.776097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d5e3e-6b0e-4b5f-be5d-7a7304f8f3e1 admin admin] PATH_INFO: `/v3/projects/6482642e77af4f5ab87f3dabd5121982/users/65df0e9f09f24ed9b58300135d0b203f/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.792688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d5e3e-6b0e-4b5f-be5d-7a7304f8f3e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6482642e77af4f5ab87f3dabd5121982, user_id=65df0e9f09f24ed9b58300135d0b203f, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:41.795548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d5e3e-6b0e-4b5f-be5d-7a7304f8f3e1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:41.813156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1088/2185] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:41 2022] PUT /identity/v3/projects/6482642e77af4f5ab87f3dabd5121982/users/65df0e9f09f24ed9b58300135d0b203f/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:41.825391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75da409-3205-42d3-947a-ad1b5d9c672f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:41.825801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75da409-3205-42d3-947a-ad1b5d9c672f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:41.826088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75da409-3205-42d3-947a-ad1b5d9c672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:41.856265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f75da409-3205-42d3-947a-ad1b5d9c672f None None] MFA Rules not processed for user `65df0e9f09f24ed9b58300135d0b203f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:41.857607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f75da409-3205-42d3-947a-ad1b5d9c672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:41.934656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1098/2186] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:43:41.946288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d5e3e-6b0e-4b5f-be5d-7a7304f8f3e1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:41.948146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e11d5e3e-6b0e-4b5f-be5d-7a7304f8f3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:41.963286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:42.019044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d5e3e-6b0e-4b5f-be5d-7a7304f8f3e1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:42.023791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73703daf-7b0b-476b-8533-6cc4dacd4d9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:42.024028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73703daf-7b0b-476b-8533-6cc4dacd4d9a service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:42.024306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73703daf-7b0b-476b-8533-6cc4dacd4d9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:42.024585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73703daf-7b0b-476b-8533-6cc4dacd4d9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:42.034343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73703daf-7b0b-476b-8533-6cc4dacd4d9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:42.035977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73703daf-7b0b-476b-8533-6cc4dacd4d9a service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:42.045854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1089/2187] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:44.268303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf98f18-c969-43f0-bb5d-75013f64ca38 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:44.268771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf98f18-c969-43f0-bb5d-75013f64ca38 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:44.268861 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf98f18-c969-43f0-bb5d-75013f64ca38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:44.296081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8bf98f18-c969-43f0-bb5d-75013f64ca38 None None] MFA Rules not processed for user `0d648fae81514f87b2c0dd7f494aee46`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:44.297313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8bf98f18-c969-43f0-bb5d-75013f64ca38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:44.388334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1099/2188] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:44.392777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73703daf-7b0b-476b-8533-6cc4dacd4d9a service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:44.394883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-73703daf-7b0b-476b-8533-6cc4dacd4d9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:44.505546 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73703daf-7b0b-476b-8533-6cc4dacd4d9a service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:44.510103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0177793-ac4e-4dbf-be3e-0bba04469627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:44.510784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0177793-ac4e-4dbf-be3e-0bba04469627 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:44.511072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0177793-ac4e-4dbf-be3e-0bba04469627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:44.511352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0177793-ac4e-4dbf-be3e-0bba04469627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:44.512138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0177793-ac4e-4dbf-be3e-0bba04469627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:44.514117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0177793-ac4e-4dbf-be3e-0bba04469627 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:44.552498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1090/2189] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:43:44 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:43:44.553686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf98f18-c969-43f0-bb5d-75013f64ca38 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:44.561049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:44.569000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf98f18-c969-43f0-bb5d-75013f64ca38 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:44.631944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae9291-aa12-484f-87c9-cfe7c98ae3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:44.632639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae9291-aa12-484f-87c9-cfe7c98ae3ac admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:44.632925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae9291-aa12-484f-87c9-cfe7c98ae3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:44.633204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae9291-aa12-484f-87c9-cfe7c98ae3ac admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:44.633907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae9291-aa12-484f-87c9-cfe7c98ae3ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:44.635908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae9291-aa12-484f-87c9-cfe7c98ae3ac admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:44.660750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1100/2190] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:43:44 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:43:44.666700 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0177793-ac4e-4dbf-be3e-0bba04469627 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:44.678544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0177793-ac4e-4dbf-be3e-0bba04469627 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:44.687342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad2631-0604-4848-a4bf-c837eb179417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:44.688131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad2631-0604-4848-a4bf-c837eb179417 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:44.688534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad2631-0604-4848-a4bf-c837eb179417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:44.688873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad2631-0604-4848-a4bf-c837eb179417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:44.689925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad2631-0604-4848-a4bf-c837eb179417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:44.692045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad2631-0604-4848-a4bf-c837eb179417 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:44.707733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1091/2191] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:44.708934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae9291-aa12-484f-87c9-cfe7c98ae3ac admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:44.720557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae9291-aa12-484f-87c9-cfe7c98ae3ac admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:44.726777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08502fd9-e6f6-4fca-ae32-8a639d67b236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:44.727833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08502fd9-e6f6-4fca-ae32-8a639d67b236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:44.728225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08502fd9-e6f6-4fca-ae32-8a639d67b236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:44.728607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08502fd9-e6f6-4fca-ae32-8a639d67b236 admin admin] PATH_INFO: `/v3/projects/c432d1ff42a048e6838af7b46ff7ec2f/users/c849716fb81d438d854cd2805b1aef71/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:44.754998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08502fd9-e6f6-4fca-ae32-8a639d67b236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c432d1ff42a048e6838af7b46ff7ec2f, user_id=c849716fb81d438d854cd2805b1aef71, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:44.757816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08502fd9-e6f6-4fca-ae32-8a639d67b236 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:44.781928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1101/2192] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:44 2022] PUT /identity/v3/projects/c432d1ff42a048e6838af7b46ff7ec2f/users/c849716fb81d438d854cd2805b1aef71/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:44.786003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad2631-0604-4848-a4bf-c837eb179417 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:44.797067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad2631-0604-4848-a4bf-c837eb179417 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:44.888927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c930b36-87a2-4fc0-96cd-e3afeb5574da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:44.889775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c930b36-87a2-4fc0-96cd-e3afeb5574da admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:44.890064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c930b36-87a2-4fc0-96cd-e3afeb5574da admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:44.890349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c930b36-87a2-4fc0-96cd-e3afeb5574da admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:44.891122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c930b36-87a2-4fc0-96cd-e3afeb5574da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:44.893128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c930b36-87a2-4fc0-96cd-e3afeb5574da admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:44.908618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1092/2193] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:43:44.916285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08502fd9-e6f6-4fca-ae32-8a639d67b236 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:44.940294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08502fd9-e6f6-4fca-ae32-8a639d67b236 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:44.947606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d6493-893a-4f34-9198-a9c1b0d9e41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:44.948543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d6493-893a-4f34-9198-a9c1b0d9e41d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:44.948881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d6493-893a-4f34-9198-a9c1b0d9e41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:44.949314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d6493-893a-4f34-9198-a9c1b0d9e41d admin admin] PATH_INFO: `/v3/projects/c432d1ff42a048e6838af7b46ff7ec2f/users/c849716fb81d438d854cd2805b1aef71/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:44.951598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d6493-893a-4f34-9198-a9c1b0d9e41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c432d1ff42a048e6838af7b46ff7ec2f, user_id=c849716fb81d438d854cd2805b1aef71, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:44.954282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d6493-893a-4f34-9198-a9c1b0d9e41d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:44.965456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1102/2194] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:43:44 2022] PUT /identity/v3/projects/c432d1ff42a048e6838af7b46ff7ec2f/users/c849716fb81d438d854cd2805b1aef71/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:49.467036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b1ddc-1dee-4099-8937-5ad90b6b9e35 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:49.467602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b1ddc-1dee-4099-8937-5ad90b6b9e35 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:49.467602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b1ddc-1dee-4099-8937-5ad90b6b9e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:49.496625 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-621b1ddc-1dee-4099-8937-5ad90b6b9e35 None None] MFA Rules not processed for user `c849716fb81d438d854cd2805b1aef71`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:49.497967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-621b1ddc-1dee-4099-8937-5ad90b6b9e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:49.510830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:49.577324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1093/2195] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:49.591032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d6493-893a-4f34-9198-a9c1b0d9e41d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:49.592944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-361d6493-893a-4f34-9198-a9c1b0d9e41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:49.688226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d6493-893a-4f34-9198-a9c1b0d9e41d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:49.695750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962c618-2113-4428-b7b0-7f208af6cd59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:49.696495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962c618-2113-4428-b7b0-7f208af6cd59 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:49.696955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962c618-2113-4428-b7b0-7f208af6cd59 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:49.697300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962c618-2113-4428-b7b0-7f208af6cd59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:49.778165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:43:49.784334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962c618-2113-4428-b7b0-7f208af6cd59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:49.786339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962c618-2113-4428-b7b0-7f208af6cd59 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:49.807672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1103/2196] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:51.008102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632eebf-1688-472b-93e8-4e5d27f8d35f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:51.008659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632eebf-1688-472b-93e8-4e5d27f8d35f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:51.009800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632eebf-1688-472b-93e8-4e5d27f8d35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:51.028556 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6632eebf-1688-472b-93e8-4e5d27f8d35f None None] MFA Rules not processed for user `0d648fae81514f87b2c0dd7f494aee46`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:51.030050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6632eebf-1688-472b-93e8-4e5d27f8d35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:51.037349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1094/2197] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:56.771180 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962c618-2113-4428-b7b0-7f208af6cd59 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:56.852825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962c618-2113-4428-b7b0-7f208af6cd59 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:56.857703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:56.858605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:56.858890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:56.859168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] PATH_INFO: `/v3/users/65df0e9f09f24ed9b58300135d0b203f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:56.860433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65df0e9f09f24ed9b58300135d0b203f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:56.862374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:56.911918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65df0e9f09f24ed9b58300135d0b203f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:56.912656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65df0e9f09f24ed9b58300135d0b203f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:56.924504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65df0e9f09f24ed9b58300135d0b203f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:56.931152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65df0e9f09f24ed9b58300135d0b203f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:56.940131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65df0e9f09f24ed9b58300135d0b203f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:56.946645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1104/2198] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:43:56 2022] DELETE /identity/v3/users/65df0e9f09f24ed9b58300135d0b203f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:43:56.951604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57a29a-d2c6-4f89-92fd-507e20c4d2bb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:56.952104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57a29a-d2c6-4f89-92fd-507e20c4d2bb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:56.952253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57a29a-d2c6-4f89-92fd-507e20c4d2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:56.970417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2d57a29a-d2c6-4f89-92fd-507e20c4d2bb None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:43:56.971657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2d57a29a-d2c6-4f89-92fd-507e20c4d2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:57.034783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1095/2199] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:43:57.044872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:57.046450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:57.108934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147af41-0c77-48f9-9119-6556720369a2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:57.112825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c63d-61f6-4725-a7a5-02be560e0120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:57.113480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c63d-61f6-4725-a7a5-02be560e0120 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:57.113762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c63d-61f6-4725-a7a5-02be560e0120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:57.114036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c63d-61f6-4725-a7a5-02be560e0120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:57.123090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c63d-61f6-4725-a7a5-02be560e0120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:57.124840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c63d-61f6-4725-a7a5-02be560e0120 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:57.135192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1105/2200] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:43:57.433205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57a29a-d2c6-4f89-92fd-507e20c4d2bb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:43:57.434780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2d57a29a-d2c6-4f89-92fd-507e20c4d2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:43:57.452951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57a29a-d2c6-4f89-92fd-507e20c4d2bb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:43:57.458820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:43:57.459776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:43:57.460225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:43:57.460507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] PATH_INFO: `/v3/projects/6482642e77af4f5ab87f3dabd5121982` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:43:57.461728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6482642e77af4f5ab87f3dabd5121982)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:43:57.463688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:43:57.520730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6482642e77af4f5ab87f3dabd5121982', 'request_id': 'req-0f83cd47-77ac-47f5-ad35-fff337e26fe9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:57.527104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6482642e77af4f5ab87f3dabd5121982', 'request_id': 'req-0f83cd47-77ac-47f5-ad35-fff337e26fe9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:57.533144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6482642e77af4f5ab87f3dabd5121982', 'request_id': 'req-0f83cd47-77ac-47f5-ad35-fff337e26fe9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:57.546754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] The token cache is being invalidate because project 6482642e77af4f5ab87f3dabd5121982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:43:57.547045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:57.547791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:43:57.549413 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1096/2201] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:43:57 2022] DELETE /identity/v3/projects/6482642e77af4f5ab87f3dabd5121982 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:44:02.303747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b2b88-e3d7-4e2f-bc6e-37c920cba8b7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:02.304290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b2b88-e3d7-4e2f-bc6e-37c920cba8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:02.304411 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b2b88-e3d7-4e2f-bc6e-37c920cba8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:02.325739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-657b2b88-e3d7-4e2f-bc6e-37c920cba8b7 None None] MFA Rules not processed for user `798de1406eac40b6bc5cf2f5bea4846f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:44:02.327012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-657b2b88-e3d7-4e2f-bc6e-37c920cba8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:02.336975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:44:02.385890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1106/2202] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:44:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:44:07.543571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:44:12.377938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=18 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:44:23.464509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:23.466625 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:23.534419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83cd47-77ac-47f5-ad35-fff337e26fe9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:23.539962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:23.540954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:23.541279 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:23.541597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] PATH_INFO: `/v3/users/eb3019f8ecf841c4a91f7af9295fb5de` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:23.542837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb3019f8ecf841c4a91f7af9295fb5de)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:23.544931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:23.595083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb3019f8ecf841c4a91f7af9295fb5de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:23.595761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb3019f8ecf841c4a91f7af9295fb5de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:23.608223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb3019f8ecf841c4a91f7af9295fb5de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:23.616266 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb3019f8ecf841c4a91f7af9295fb5de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:23.617102 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb3019f8ecf841c4a91f7af9295fb5de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:23.632010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1097/2203] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:44:23 2022] DELETE /identity/v3/users/eb3019f8ecf841c4a91f7af9295fb5de => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:44:23.637049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d054d86-cc5a-43fe-8d59-357671f6207f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:23.637580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d054d86-cc5a-43fe-8d59-357671f6207f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:23.637677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d054d86-cc5a-43fe-8d59-357671f6207f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:23.655266 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0d054d86-cc5a-43fe-8d59-357671f6207f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:44:23.656596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0d054d86-cc5a-43fe-8d59-357671f6207f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:23.734490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1107/2204] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:44:23.743691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:23.745103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:23.813406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8a6cd-9f58-4034-a0f0-ce10a19875b5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:23.817585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb638fa-1add-4d4e-a09f-a5b4590b1bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:23.818338 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb638fa-1add-4d4e-a09f-a5b4590b1bdf service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:23.818687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb638fa-1add-4d4e-a09f-a5b4590b1bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:23.819400 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb638fa-1add-4d4e-a09f-a5b4590b1bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:23.838471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb638fa-1add-4d4e-a09f-a5b4590b1bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:23.840236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb638fa-1add-4d4e-a09f-a5b4590b1bdf service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:23.852906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1098/2205] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:44:24.162725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d054d86-cc5a-43fe-8d59-357671f6207f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:24.164442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0d054d86-cc5a-43fe-8d59-357671f6207f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:24.185792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d054d86-cc5a-43fe-8d59-357671f6207f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:24.201226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:24.202036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.202319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.202596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] PATH_INFO: `/v3/projects/054bd78b264946b59cfe5911daf1868e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.204431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=054bd78b264946b59cfe5911daf1868e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:24.205997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:24.265820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '054bd78b264946b59cfe5911daf1868e', 'request_id': 'req-0a356926-2bdc-431f-98cb-8aa0929ba589'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:24.279990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '054bd78b264946b59cfe5911daf1868e', 'request_id': 'req-0a356926-2bdc-431f-98cb-8aa0929ba589'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:24.289515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '054bd78b264946b59cfe5911daf1868e', 'request_id': 'req-0a356926-2bdc-431f-98cb-8aa0929ba589'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:24.304965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] The token cache is being invalidate because project 054bd78b264946b59cfe5911daf1868e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:44:24.305365 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:24.306180 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a356926-2bdc-431f-98cb-8aa0929ba589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:44:24.308624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1108/2206] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:44:24 2022] DELETE /identity/v3/projects/054bd78b264946b59cfe5911daf1868e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:44:24.313664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c9622-9798-4b63-914b-d1bd57b7d0bf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.313954 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c9622-9798-4b63-914b-d1bd57b7d0bf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.314231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c9622-9798-4b63-914b-d1bd57b7d0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.334051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e95c9622-9798-4b63-914b-d1bd57b7d0bf None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:44:24.335552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e95c9622-9798-4b63-914b-d1bd57b7d0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:24.355886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:44:24.416142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1099/2207] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:44:24.424168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a69f48-e053-44ac-9a3d-b457a8e707af None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.424532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a69f48-e053-44ac-9a3d-b457a8e707af None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.424752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a69f48-e053-44ac-9a3d-b457a8e707af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.447244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-52a69f48-e053-44ac-9a3d-b457a8e707af None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:44:24.449073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-52a69f48-e053-44ac-9a3d-b457a8e707af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:24.458533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1109/2208] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:44:24.462335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c9622-9798-4b63-914b-d1bd57b7d0bf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:24.476211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c9622-9798-4b63-914b-d1bd57b7d0bf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:24.489771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868ab74-69c9-4d2e-a1fc-b753a34b247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:24.490457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868ab74-69c9-4d2e-a1fc-b753a34b247f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.490771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868ab74-69c9-4d2e-a1fc-b753a34b247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.491001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868ab74-69c9-4d2e-a1fc-b753a34b247f admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.491803 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868ab74-69c9-4d2e-a1fc-b753a34b247f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:44:24.492118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868ab74-69c9-4d2e-a1fc-b753a34b247f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:24.493757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868ab74-69c9-4d2e-a1fc-b753a34b247f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:24.507501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1100/2209] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:44:24.511407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a69f48-e053-44ac-9a3d-b457a8e707af None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:24.526867 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a69f48-e053-44ac-9a3d-b457a8e707af None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:24.532558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76445ac0-1c13-4834-8641-c1649688d9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:24.533244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76445ac0-1c13-4834-8641-c1649688d9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.533527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76445ac0-1c13-4834-8641-c1649688d9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.533857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76445ac0-1c13-4834-8641-c1649688d9b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.534559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76445ac0-1c13-4834-8641-c1649688d9b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:24.538541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76445ac0-1c13-4834-8641-c1649688d9b6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:24.595654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1110/2210] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:44:24 2022] POST /identity/v3/projects => generated 363 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:44:24.599823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868ab74-69c9-4d2e-a1fc-b753a34b247f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:24.611152 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868ab74-69c9-4d2e-a1fc-b753a34b247f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:24.696644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dba384-33b8-4fa4-8253-bb79672e5d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:24.697325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dba384-33b8-4fa4-8253-bb79672e5d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.697609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dba384-33b8-4fa4-8253-bb79672e5d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.697886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dba384-33b8-4fa4-8253-bb79672e5d82 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.698581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dba384-33b8-4fa4-8253-bb79672e5d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:24.700611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dba384-33b8-4fa4-8253-bb79672e5d82 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:24.729144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1101/2211] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:44:24 2022] POST /identity/v3/users => generated 282 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:44:24.730027 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76445ac0-1c13-4834-8641-c1649688d9b6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:24.743227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76445ac0-1c13-4834-8641-c1649688d9b6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:24.748654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e8d57-62dd-41c3-bf04-51d494e3a48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:24.749530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e8d57-62dd-41c3-bf04-51d494e3a48b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.749661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e8d57-62dd-41c3-bf04-51d494e3a48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.749941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e8d57-62dd-41c3-bf04-51d494e3a48b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.750673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e8d57-62dd-41c3-bf04-51d494e3a48b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:24.752416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e8d57-62dd-41c3-bf04-51d494e3a48b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:24.770753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1111/2212] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:44:24.771982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dba384-33b8-4fa4-8253-bb79672e5d82 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:24.782549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dba384-33b8-4fa4-8253-bb79672e5d82 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:24.787736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208c7a4-db1a-4fd2-8867-8e0a9294a5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:24.788447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208c7a4-db1a-4fd2-8867-8e0a9294a5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.788794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208c7a4-db1a-4fd2-8867-8e0a9294a5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.789125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208c7a4-db1a-4fd2-8867-8e0a9294a5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.789928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208c7a4-db1a-4fd2-8867-8e0a9294a5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:24.791649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208c7a4-db1a-4fd2-8867-8e0a9294a5e1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:24.806684 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1102/2213] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:44:24.807766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e8d57-62dd-41c3-bf04-51d494e3a48b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:24.819720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e8d57-62dd-41c3-bf04-51d494e3a48b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:24.825116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bcfcc-5d8f-44c0-80e0-91bc4f7576da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:24.826167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bcfcc-5d8f-44c0-80e0-91bc4f7576da admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.826468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bcfcc-5d8f-44c0-80e0-91bc4f7576da admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.826777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bcfcc-5d8f-44c0-80e0-91bc4f7576da admin admin] PATH_INFO: `/v3/projects/9e6df83b1eac49aa9d929eccbd99142c/users/9509934717c94a07b8ddcb228c462b56/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.843608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0bcfcc-5d8f-44c0-80e0-91bc4f7576da admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6df83b1eac49aa9d929eccbd99142c, user_id=9509934717c94a07b8ddcb228c462b56, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:24.846491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0bcfcc-5d8f-44c0-80e0-91bc4f7576da admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:24.860078 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1112/2214] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:44:24 2022] PUT /identity/v3/projects/9e6df83b1eac49aa9d929eccbd99142c/users/9509934717c94a07b8ddcb228c462b56/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:44:24.871396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0d86d-0ff7-42a6-a5a3-78cd1bedefa0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:24.871720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0d86d-0ff7-42a6-a5a3-78cd1bedefa0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:24.872009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0d86d-0ff7-42a6-a5a3-78cd1bedefa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:24.902994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d3d0d86d-0ff7-42a6-a5a3-78cd1bedefa0 None None] MFA Rules not processed for user `9509934717c94a07b8ddcb228c462b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:44:24.904271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d3d0d86d-0ff7-42a6-a5a3-78cd1bedefa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:24.975268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1103/2215] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:44:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:44:24.987074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bcfcc-5d8f-44c0-80e0-91bc4f7576da admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:24.988937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7e0bcfcc-5d8f-44c0-80e0-91bc4f7576da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:25.012508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:44:25.059883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bcfcc-5d8f-44c0-80e0-91bc4f7576da admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:25.069117 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea351b10-0ef1-40b4-8934-450d72dca849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:25.069788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea351b10-0ef1-40b4-8934-450d72dca849 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:25.070072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea351b10-0ef1-40b4-8934-450d72dca849 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:25.070349 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea351b10-0ef1-40b4-8934-450d72dca849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:25.080051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea351b10-0ef1-40b4-8934-450d72dca849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:25.081521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea351b10-0ef1-40b4-8934-450d72dca849 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:25.092236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1113/2216] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:44:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:44:34.972415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:44:35.086742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=17 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:44:45.978248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0d86d-0ff7-42a6-a5a3-78cd1bedefa0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:44:45.979936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d3d0d86d-0ff7-42a6-a5a3-78cd1bedefa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:46.040010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0d86d-0ff7-42a6-a5a3-78cd1bedefa0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:44:46.044189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:44:46.044846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:44:46.045135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:44:46.045418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:44:46.047425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:44:46.113764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:44:46.115226 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:44:46.125723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1104/2217] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:44:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:44:56.121049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=50 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:09.247681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791128-e181-451c-bf0a-da9e96fb8102 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:09.248191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791128-e181-451c-bf0a-da9e96fb8102 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:09.248450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791128-e181-451c-bf0a-da9e96fb8102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:09.271856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6f791128-e181-451c-bf0a-da9e96fb8102 None None] MFA Rules not processed for user `0b22e96941dd414da4fa22ed4bd36984`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:09.273354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6f791128-e181-451c-bf0a-da9e96fb8102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:09.348595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1114/2218] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:10.795225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:10.796857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:10.873869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df1bb2-1d4d-4953-b99d-50219050f33c service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:10.879919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84f54e-6294-4bff-8388-e5400983bee7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:10.880597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f54e-6294-4bff-8388-e5400983bee7 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:10.880902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f54e-6294-4bff-8388-e5400983bee7 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:10.881169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f54e-6294-4bff-8388-e5400983bee7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:10.883073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ef84f54e-6294-4bff-8388-e5400983bee7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:10.895035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84f54e-6294-4bff-8388-e5400983bee7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:10.896773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84f54e-6294-4bff-8388-e5400983bee7 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:10.907147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1105/2219] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:45:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:45:14.094441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f791128-e181-451c-bf0a-da9e96fb8102 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:14.162684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f791128-e181-451c-bf0a-da9e96fb8102 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:14.168604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:14.169498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:14.169813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:14.170094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] PATH_INFO: `/v3/users/9509934717c94a07b8ddcb228c462b56` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:14.171414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9509934717c94a07b8ddcb228c462b56)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:14.173407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:14.232351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9509934717c94a07b8ddcb228c462b56'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.233058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9509934717c94a07b8ddcb228c462b56'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.246967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9509934717c94a07b8ddcb228c462b56'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.255830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9509934717c94a07b8ddcb228c462b56'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.267718 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9509934717c94a07b8ddcb228c462b56'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.274153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1115/2220] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:45:14 2022] DELETE /identity/v3/users/9509934717c94a07b8ddcb228c462b56 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:45:14.279065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f99860-0523-42f7-bccd-ae997d776df3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:14.279591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f99860-0523-42f7-bccd-ae997d776df3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:14.279686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f99860-0523-42f7-bccd-ae997d776df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:14.305607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c7f99860-0523-42f7-bccd-ae997d776df3 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:14.307350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c7f99860-0523-42f7-bccd-ae997d776df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:14.373083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1106/2221] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:14.382872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:14.384511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:14.412676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:14.447132 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41318bed-a761-418a-833c-072ad98b42fa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:14.451663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf7ae6-16e4-4da6-942d-116026d11b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:14.452328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf7ae6-16e4-4da6-942d-116026d11b87 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:14.452616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf7ae6-16e4-4da6-942d-116026d11b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:14.452894 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf7ae6-16e4-4da6-942d-116026d11b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:14.462609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf7ae6-16e4-4da6-942d-116026d11b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:14.464124 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf7ae6-16e4-4da6-942d-116026d11b87 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:14.474377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1116/2222] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:45:14.746636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f99860-0523-42f7-bccd-ae997d776df3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:14.748446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c7f99860-0523-42f7-bccd-ae997d776df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:14.761284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f99860-0523-42f7-bccd-ae997d776df3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:14.766213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:14.767096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:14.767377 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:14.767683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] PATH_INFO: `/v3/projects/9e6df83b1eac49aa9d929eccbd99142c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:14.768845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e6df83b1eac49aa9d929eccbd99142c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:14.770904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:14.820907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e6df83b1eac49aa9d929eccbd99142c', 'request_id': 'req-09685d69-7cbf-45e4-903a-74d0fade8d89'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.827096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e6df83b1eac49aa9d929eccbd99142c', 'request_id': 'req-09685d69-7cbf-45e4-903a-74d0fade8d89'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.833404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e6df83b1eac49aa9d929eccbd99142c', 'request_id': 'req-09685d69-7cbf-45e4-903a-74d0fade8d89'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.847262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] The token cache is being invalidate because project 9e6df83b1eac49aa9d929eccbd99142c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:45:14.847586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.848334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09685d69-7cbf-45e4-903a-74d0fade8d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:14.850126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1107/2223] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:45:14 2022] DELETE /identity/v3/projects/9e6df83b1eac49aa9d929eccbd99142c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:45:14.855101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b855f3-474c-425e-8c81-8a34fc2f69c1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:14.855415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b855f3-474c-425e-8c81-8a34fc2f69c1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:14.855760 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b855f3-474c-425e-8c81-8a34fc2f69c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:14.874073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-21b855f3-474c-425e-8c81-8a34fc2f69c1 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:14.875353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-21b855f3-474c-425e-8c81-8a34fc2f69c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:14.938693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1117/2224] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:14.947090 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e97540-407a-4da6-a4d3-83333f24d9ce None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:14.947382 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e97540-407a-4da6-a4d3-83333f24d9ce None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:14.947682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e97540-407a-4da6-a4d3-83333f24d9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:14.970364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-14e97540-407a-4da6-a4d3-83333f24d9ce None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:14.971581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-14e97540-407a-4da6-a4d3-83333f24d9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:14.979491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1108/2225] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:14.983728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b855f3-474c-425e-8c81-8a34fc2f69c1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:14.993449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b855f3-474c-425e-8c81-8a34fc2f69c1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:14.998157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee236922-5543-4b0b-bd77-fc5e1e145976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:14.998816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee236922-5543-4b0b-bd77-fc5e1e145976 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:14.999110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee236922-5543-4b0b-bd77-fc5e1e145976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:14.999392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee236922-5543-4b0b-bd77-fc5e1e145976 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:15.000159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee236922-5543-4b0b-bd77-fc5e1e145976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:45:15.000471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee236922-5543-4b0b-bd77-fc5e1e145976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:15.002100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee236922-5543-4b0b-bd77-fc5e1e145976 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:15.013725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1118/2226] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:45:15.018020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e97540-407a-4da6-a4d3-83333f24d9ce None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:15.027226 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e97540-407a-4da6-a4d3-83333f24d9ce None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:15.032012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c9408-5a28-4e26-9c2d-5a9cb61e7c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:15.032665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c9408-5a28-4e26-9c2d-5a9cb61e7c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:15.032929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c9408-5a28-4e26-9c2d-5a9cb61e7c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:15.033229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c9408-5a28-4e26-9c2d-5a9cb61e7c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:15.033937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c9408-5a28-4e26-9c2d-5a9cb61e7c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:15.035953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c9408-5a28-4e26-9c2d-5a9cb61e7c49 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:15.039353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:15.070336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1109/2227] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:45:15 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:45:15.071599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee236922-5543-4b0b-bd77-fc5e1e145976 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:15.081859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee236922-5543-4b0b-bd77-fc5e1e145976 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:15.139405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd49b89-9570-423a-a637-c5449ed192b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:15.140042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd49b89-9570-423a-a637-c5449ed192b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:15.140321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd49b89-9570-423a-a637-c5449ed192b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:15.140604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd49b89-9570-423a-a637-c5449ed192b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:15.141302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd49b89-9570-423a-a637-c5449ed192b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:15.143258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd49b89-9570-423a-a637-c5449ed192b5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:15.164574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1119/2228] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:45:15 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:45:15.168887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c9408-5a28-4e26-9c2d-5a9cb61e7c49 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:15.179420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c9408-5a28-4e26-9c2d-5a9cb61e7c49 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:15.184242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6407f-7981-409b-9549-4f36c52f8879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:15.184903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6407f-7981-409b-9549-4f36c52f8879 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:15.185182 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6407f-7981-409b-9549-4f36c52f8879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:15.185475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6407f-7981-409b-9549-4f36c52f8879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:15.186208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6407f-7981-409b-9549-4f36c52f8879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:15.187873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6407f-7981-409b-9549-4f36c52f8879 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:15.198378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1110/2229] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:45:15.202396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd49b89-9570-423a-a637-c5449ed192b5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:15.213178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd49b89-9570-423a-a637-c5449ed192b5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:15.218715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bff56-f8b1-441a-b6fe-77f32ee7a5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:15.219388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bff56-f8b1-441a-b6fe-77f32ee7a5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:15.219713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bff56-f8b1-441a-b6fe-77f32ee7a5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:15.220000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bff56-f8b1-441a-b6fe-77f32ee7a5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:15.220735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bff56-f8b1-441a-b6fe-77f32ee7a5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:15.222363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bff56-f8b1-441a-b6fe-77f32ee7a5cf admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:15.232722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1120/2230] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:45:15.236925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6407f-7981-409b-9549-4f36c52f8879 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:15.250344 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6407f-7981-409b-9549-4f36c52f8879 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:15.255433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c8f42-6a69-48c2-ac45-d61f9a2e3a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:15.256442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c8f42-6a69-48c2-ac45-d61f9a2e3a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:15.256749 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c8f42-6a69-48c2-ac45-d61f9a2e3a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:15.257026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c8f42-6a69-48c2-ac45-d61f9a2e3a20 admin admin] PATH_INFO: `/v3/projects/2308e88a59de4697a5d1e4816ced05ac/users/205a1445b0db4b058cdfc60cac215d03/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:15.272446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c8f42-6a69-48c2-ac45-d61f9a2e3a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2308e88a59de4697a5d1e4816ced05ac, user_id=205a1445b0db4b058cdfc60cac215d03, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:15.275092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c8f42-6a69-48c2-ac45-d61f9a2e3a20 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:15.291569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1111/2231] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:45:15 2022] PUT /identity/v3/projects/2308e88a59de4697a5d1e4816ced05ac/users/205a1445b0db4b058cdfc60cac215d03/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:45:15.306642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5933756-2dcb-48f8-84ff-6bae5de36fe6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:15.306944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5933756-2dcb-48f8-84ff-6bae5de36fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:15.307229 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5933756-2dcb-48f8-84ff-6bae5de36fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:15.336325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a5933756-2dcb-48f8-84ff-6bae5de36fe6 None None] MFA Rules not processed for user `205a1445b0db4b058cdfc60cac215d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:15.337603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a5933756-2dcb-48f8-84ff-6bae5de36fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:15.357632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:15.403336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1121/2232] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:15.414817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c8f42-6a69-48c2-ac45-d61f9a2e3a20 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:15.416634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-461c8f42-6a69-48c2-ac45-d61f9a2e3a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:15.480243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c8f42-6a69-48c2-ac45-d61f9a2e3a20 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:15.488478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0725112-fb3b-44d9-bc79-5b3d04288794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:15.488478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0725112-fb3b-44d9-bc79-5b3d04288794 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:15.488478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0725112-fb3b-44d9-bc79-5b3d04288794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:15.488812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0725112-fb3b-44d9-bc79-5b3d04288794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:15.500472 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0725112-fb3b-44d9-bc79-5b3d04288794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:15.502154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0725112-fb3b-44d9-bc79-5b3d04288794 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:15.512989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1112/2233] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:45:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:45:25.400020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=17 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:25.507103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:26.408890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5933756-2dcb-48f8-84ff-6bae5de36fe6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:26.478267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5933756-2dcb-48f8-84ff-6bae5de36fe6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:26.484093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:26.485185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:26.485512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:26.485829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] PATH_INFO: `/v3/users/205a1445b0db4b058cdfc60cac215d03` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:26.487145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=205a1445b0db4b058cdfc60cac215d03)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:26.489759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:26.541778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205a1445b0db4b058cdfc60cac215d03'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:26.542716 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205a1445b0db4b058cdfc60cac215d03'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:26.555135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205a1445b0db4b058cdfc60cac215d03'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:26.563262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205a1445b0db4b058cdfc60cac215d03'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:26.573199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205a1445b0db4b058cdfc60cac215d03'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:26.579368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1122/2234] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:45:26 2022] DELETE /identity/v3/users/205a1445b0db4b058cdfc60cac215d03 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:45:26.584141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec1907-5385-4a3e-85b2-8cc85758d3e2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:26.584640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec1907-5385-4a3e-85b2-8cc85758d3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:26.584727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec1907-5385-4a3e-85b2-8cc85758d3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:26.602121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d2ec1907-5385-4a3e-85b2-8cc85758d3e2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:26.603493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d2ec1907-5385-4a3e-85b2-8cc85758d3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:26.662879 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1113/2235] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:26.672650 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:26.674354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:26.740939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ad5d-acc0-4c0e-b929-c80e535be3a2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:26.745426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6290e02-8934-48be-84fd-84b1a5afc104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:26.746081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6290e02-8934-48be-84fd-84b1a5afc104 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:26.746395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6290e02-8934-48be-84fd-84b1a5afc104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:26.746639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6290e02-8934-48be-84fd-84b1a5afc104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:26.756380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6290e02-8934-48be-84fd-84b1a5afc104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:26.757960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6290e02-8934-48be-84fd-84b1a5afc104 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:26.767847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1123/2236] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:45:27.108348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec1907-5385-4a3e-85b2-8cc85758d3e2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:27.109931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d2ec1907-5385-4a3e-85b2-8cc85758d3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:27.123412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec1907-5385-4a3e-85b2-8cc85758d3e2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:27.129626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:27.130514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.130789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.131178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] PATH_INFO: `/v3/projects/2308e88a59de4697a5d1e4816ced05ac` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.132285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2308e88a59de4697a5d1e4816ced05ac)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:27.134300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:27.186797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2308e88a59de4697a5d1e4816ced05ac', 'request_id': 'req-d9a094c6-b453-4b9f-ad81-3878777e2ea1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:27.193539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2308e88a59de4697a5d1e4816ced05ac', 'request_id': 'req-d9a094c6-b453-4b9f-ad81-3878777e2ea1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:27.200128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2308e88a59de4697a5d1e4816ced05ac', 'request_id': 'req-d9a094c6-b453-4b9f-ad81-3878777e2ea1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:27.212825 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] The token cache is being invalidate because project 2308e88a59de4697a5d1e4816ced05ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:45:27.213066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:27.213753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d9a094c6-b453-4b9f-ad81-3878777e2ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:45:27.215305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1114/2237] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:45:27 2022] DELETE /identity/v3/projects/2308e88a59de4697a5d1e4816ced05ac => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:45:27.220543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7101e-cb2b-48c9-acdf-aa3965605545 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.220952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7101e-cb2b-48c9-acdf-aa3965605545 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.221295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7101e-cb2b-48c9-acdf-aa3965605545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.253188 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-67b7101e-cb2b-48c9-acdf-aa3965605545 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:27.255562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-67b7101e-cb2b-48c9-acdf-aa3965605545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:27.274118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:27.334823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1124/2238] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:27.345232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8460e-63c1-498b-bd6d-061a11c2c1dd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.345601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8460e-63c1-498b-bd6d-061a11c2c1dd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.345946 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8460e-63c1-498b-bd6d-061a11c2c1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.365583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0ca8460e-63c1-498b-bd6d-061a11c2c1dd None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:27.366805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0ca8460e-63c1-498b-bd6d-061a11c2c1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:27.385233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1115/2239] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:27.389639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7101e-cb2b-48c9-acdf-aa3965605545 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:27.408687 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7101e-cb2b-48c9-acdf-aa3965605545 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:27.413295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb17f4-0ebe-4db0-abdf-e7726b791fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:27.413992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb17f4-0ebe-4db0-abdf-e7726b791fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.414274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb17f4-0ebe-4db0-abdf-e7726b791fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.414550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb17f4-0ebe-4db0-abdf-e7726b791fe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.415295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb17f4-0ebe-4db0-abdf-e7726b791fe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:45:27.415618 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb17f4-0ebe-4db0-abdf-e7726b791fe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:27.417270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb17f4-0ebe-4db0-abdf-e7726b791fe0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:27.429456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1125/2240] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:45:27.433750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8460e-63c1-498b-bd6d-061a11c2c1dd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:27.443176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8460e-63c1-498b-bd6d-061a11c2c1dd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:27.447811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa8aa5-2188-456e-be2e-5a1b1e8112a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:27.448469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa8aa5-2188-456e-be2e-5a1b1e8112a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.448752 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa8aa5-2188-456e-be2e-5a1b1e8112a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.449028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa8aa5-2188-456e-be2e-5a1b1e8112a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.449734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa8aa5-2188-456e-be2e-5a1b1e8112a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:27.451703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa8aa5-2188-456e-be2e-5a1b1e8112a5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:27.478960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1116/2241] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:45:27 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:45:27.483270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb17f4-0ebe-4db0-abdf-e7726b791fe0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:27.493889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb17f4-0ebe-4db0-abdf-e7726b791fe0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:27.550823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab1b61-bc55-400f-92ed-28c82f2b3f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:27.551570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab1b61-bc55-400f-92ed-28c82f2b3f5a admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.551863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab1b61-bc55-400f-92ed-28c82f2b3f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.552145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab1b61-bc55-400f-92ed-28c82f2b3f5a admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.552840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab1b61-bc55-400f-92ed-28c82f2b3f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:27.554834 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab1b61-bc55-400f-92ed-28c82f2b3f5a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:27.576112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1126/2242] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:45:27 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:45:27.580301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa8aa5-2188-456e-be2e-5a1b1e8112a5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:27.591233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa8aa5-2188-456e-be2e-5a1b1e8112a5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:27.596472 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2a260-f5ce-4d34-aebd-a873037bbafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:27.597159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2a260-f5ce-4d34-aebd-a873037bbafd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.597431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2a260-f5ce-4d34-aebd-a873037bbafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.597716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2a260-f5ce-4d34-aebd-a873037bbafd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.598492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2a260-f5ce-4d34-aebd-a873037bbafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:27.600153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2a260-f5ce-4d34-aebd-a873037bbafd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:27.611092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1117/2243] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:45:27.615126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab1b61-bc55-400f-92ed-28c82f2b3f5a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:27.625677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab1b61-bc55-400f-92ed-28c82f2b3f5a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:27.635274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea828e-b5f3-4fbf-aad3-18d7e69de0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:27.635977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea828e-b5f3-4fbf-aad3-18d7e69de0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.636262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea828e-b5f3-4fbf-aad3-18d7e69de0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.636538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea828e-b5f3-4fbf-aad3-18d7e69de0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.637264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea828e-b5f3-4fbf-aad3-18d7e69de0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:27.638899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea828e-b5f3-4fbf-aad3-18d7e69de0a3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:27.653057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1127/2244] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:45:27.654696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2a260-f5ce-4d34-aebd-a873037bbafd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:27.665671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2a260-f5ce-4d34-aebd-a873037bbafd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:27.670710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24f389-3c07-4bf0-a919-14285ed43dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:27.671637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24f389-3c07-4bf0-a919-14285ed43dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.671906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24f389-3c07-4bf0-a919-14285ed43dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.672205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24f389-3c07-4bf0-a919-14285ed43dd9 admin admin] PATH_INFO: `/v3/projects/10b8c7816f7f467ebec2809ec0a3fe7b/users/104a9c10adde46828fb8592a095e86df/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.688059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24f389-3c07-4bf0-a919-14285ed43dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b8c7816f7f467ebec2809ec0a3fe7b, user_id=104a9c10adde46828fb8592a095e86df, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:27.691063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24f389-3c07-4bf0-a919-14285ed43dd9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:27.705167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1118/2245] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:45:27 2022] PUT /identity/v3/projects/10b8c7816f7f467ebec2809ec0a3fe7b/users/104a9c10adde46828fb8592a095e86df/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:45:27.710337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5370701-8cd9-433b-a88f-bf2b7b4e6289 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.710630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5370701-8cd9-433b-a88f-bf2b7b4e6289 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.710907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5370701-8cd9-433b-a88f-bf2b7b4e6289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.728773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c5370701-8cd9-433b-a88f-bf2b7b4e6289 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:27.729953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c5370701-8cd9-433b-a88f-bf2b7b4e6289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:27.790746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1128/2246] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:45:27.800345 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24f389-3c07-4bf0-a919-14285ed43dd9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:27.802507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aa24f389-3c07-4bf0-a919-14285ed43dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:27.825784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:27.861959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24f389-3c07-4bf0-a919-14285ed43dd9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:27.866403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97269080-8072-438b-bd45-2fd293442470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:27.867047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97269080-8072-438b-bd45-2fd293442470 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:27.867329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97269080-8072-438b-bd45-2fd293442470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:27.867643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97269080-8072-438b-bd45-2fd293442470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:27.877416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97269080-8072-438b-bd45-2fd293442470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:27.878787 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97269080-8072-438b-bd45-2fd293442470 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:27.888878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1119/2247] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:45:32.002396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89468-2ab4-4649-a967-2d5d8eb33c1d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:32.002873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89468-2ab4-4649-a967-2d5d8eb33c1d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:32.002952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89468-2ab4-4649-a967-2d5d8eb33c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:32.030926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-50a89468-2ab4-4649-a967-2d5d8eb33c1d None None] MFA Rules not processed for user `104a9c10adde46828fb8592a095e86df`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:32.032284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-50a89468-2ab4-4649-a967-2d5d8eb33c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:32.058640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:32.111113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1129/2248] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:32.115741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97269080-8072-438b-bd45-2fd293442470 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:32.133756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97269080-8072-438b-bd45-2fd293442470 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:32.139374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab158b03-6462-49cc-b779-c771a4d0969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:32.140092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab158b03-6462-49cc-b779-c771a4d0969a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:32.140380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab158b03-6462-49cc-b779-c771a4d0969a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:32.140660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab158b03-6462-49cc-b779-c771a4d0969a admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:32.141379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab158b03-6462-49cc-b779-c771a4d0969a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:32.143350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab158b03-6462-49cc-b779-c771a4d0969a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:32.177300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1120/2249] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:45:32 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:45:32.178100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89468-2ab4-4649-a967-2d5d8eb33c1d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:32.190969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89468-2ab4-4649-a967-2d5d8eb33c1d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:32.253311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19215424-6327-4911-a46a-c8ad44f069e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:32.254097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215424-6327-4911-a46a-c8ad44f069e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:32.254389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215424-6327-4911-a46a-c8ad44f069e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:32.254671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215424-6327-4911-a46a-c8ad44f069e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:32.255367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19215424-6327-4911-a46a-c8ad44f069e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:32.257354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19215424-6327-4911-a46a-c8ad44f069e1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:32.278030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1130/2250] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:45:32 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:45:32.281858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab158b03-6462-49cc-b779-c771a4d0969a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:32.292585 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab158b03-6462-49cc-b779-c771a4d0969a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:32.297712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c2ca5-afed-42ed-8c97-3213a7062d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:32.298487 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c2ca5-afed-42ed-8c97-3213a7062d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:32.298874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c2ca5-afed-42ed-8c97-3213a7062d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:32.299286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c2ca5-afed-42ed-8c97-3213a7062d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:32.300154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c2ca5-afed-42ed-8c97-3213a7062d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:32.302214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c2ca5-afed-42ed-8c97-3213a7062d02 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:32.313952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1121/2251] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:45:32.318775 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19215424-6327-4911-a46a-c8ad44f069e1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:32.347163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19215424-6327-4911-a46a-c8ad44f069e1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:32.357709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35687f07-b2dd-414f-b2f5-daff4e232add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:32.358717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687f07-b2dd-414f-b2f5-daff4e232add admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:32.359061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687f07-b2dd-414f-b2f5-daff4e232add admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:32.359345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687f07-b2dd-414f-b2f5-daff4e232add admin admin] PATH_INFO: `/v3/projects/5f17a3c1e2e740279814ec541d2b952e/users/60e4ad98be134148b8ae378ed4f84425/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:32.374335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35687f07-b2dd-414f-b2f5-daff4e232add admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f17a3c1e2e740279814ec541d2b952e, user_id=60e4ad98be134148b8ae378ed4f84425, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:32.377130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35687f07-b2dd-414f-b2f5-daff4e232add admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:32.394183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1131/2252] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:45:32 2022] PUT /identity/v3/projects/5f17a3c1e2e740279814ec541d2b952e/users/60e4ad98be134148b8ae378ed4f84425/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:45:32.398485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c2ca5-afed-42ed-8c97-3213a7062d02 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:32.409185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c2ca5-afed-42ed-8c97-3213a7062d02 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:32.485283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f18d4-1199-44c9-9247-f38f005cb50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:32.487050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f18d4-1199-44c9-9247-f38f005cb50e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:32.487502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f18d4-1199-44c9-9247-f38f005cb50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:32.487919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f18d4-1199-44c9-9247-f38f005cb50e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:32.489714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f18d4-1199-44c9-9247-f38f005cb50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:32.491436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f18d4-1199-44c9-9247-f38f005cb50e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:32.502628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1122/2253] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:45:32.506968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35687f07-b2dd-414f-b2f5-daff4e232add admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:32.518532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35687f07-b2dd-414f-b2f5-daff4e232add admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:32.523339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad020d-c0da-4ff3-a833-b902bbfcf857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:32.524373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad020d-c0da-4ff3-a833-b902bbfcf857 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:32.524733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad020d-c0da-4ff3-a833-b902bbfcf857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:32.525070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad020d-c0da-4ff3-a833-b902bbfcf857 admin admin] PATH_INFO: `/v3/projects/5f17a3c1e2e740279814ec541d2b952e/users/60e4ad98be134148b8ae378ed4f84425/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:32.527355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad020d-c0da-4ff3-a833-b902bbfcf857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f17a3c1e2e740279814ec541d2b952e, user_id=60e4ad98be134148b8ae378ed4f84425, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:32.530065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad020d-c0da-4ff3-a833-b902bbfcf857 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:32.541684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1132/2254] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:45:32 2022] PUT /identity/v3/projects/5f17a3c1e2e740279814ec541d2b952e/users/60e4ad98be134148b8ae378ed4f84425/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:45:33.668156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3565dde-57bf-481e-9077-f7fb3ae675a4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:33.669103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3565dde-57bf-481e-9077-f7fb3ae675a4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:33.669758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3565dde-57bf-481e-9077-f7fb3ae675a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:33.696476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e3565dde-57bf-481e-9077-f7fb3ae675a4 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:33.698083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e3565dde-57bf-481e-9077-f7fb3ae675a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:33.792166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1123/2255] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:45:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:33.844770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad020d-c0da-4ff3-a833-b902bbfcf857 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:33.848987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-67ad020d-c0da-4ff3-a833-b902bbfcf857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:33.935338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad020d-c0da-4ff3-a833-b902bbfcf857 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:33.943306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65940373-206f-4f7b-b38f-622ab01f7d86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:33.944323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65940373-206f-4f7b-b38f-622ab01f7d86 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:33.944768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65940373-206f-4f7b-b38f-622ab01f7d86 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:33.945235 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65940373-206f-4f7b-b38f-622ab01f7d86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:33.955575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65940373-206f-4f7b-b38f-622ab01f7d86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:33.957650 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65940373-206f-4f7b-b38f-622ab01f7d86 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:33.963629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:33.968838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1133/2256] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:45:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:45:38.662581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a18ee4-9032-41aa-9701-c54871c14b95 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:38.663595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a18ee4-9032-41aa-9701-c54871c14b95 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:38.664179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a18ee4-9032-41aa-9701-c54871c14b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:38.718473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-30a18ee4-9032-41aa-9701-c54871c14b95 None None] MFA Rules not processed for user `60e4ad98be134148b8ae378ed4f84425`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:38.720061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-30a18ee4-9032-41aa-9701-c54871c14b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:38.742324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:38.822961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1124/2257] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:38.830515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65940373-206f-4f7b-b38f-622ab01f7d86 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:38.832328 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-65940373-206f-4f7b-b38f-622ab01f7d86 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:38.929212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65940373-206f-4f7b-b38f-622ab01f7d86 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:38.935796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f8b19-71f3-4a30-ba4e-9b44cc5bd3d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc21faf2237d405bba03e5309d93629f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:38.936552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f8b19-71f3-4a30-ba4e-9b44cc5bd3d6 service glance] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:38.936996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f8b19-71f3-4a30-ba4e-9b44cc5bd3d6 service glance] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:38.937260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f8b19-71f3-4a30-ba4e-9b44cc5bd3d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:39.039013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f8b19-71f3-4a30-ba4e-9b44cc5bd3d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:39.050571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f8b19-71f3-4a30-ba4e-9b44cc5bd3d6 service glance] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:39.064844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1134/2258] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:45:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:45:39.335691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21996984-a98b-45ac-b554-05c77f4cc12b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:39.336339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21996984-a98b-45ac-b554-05c77f4cc12b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:39.337054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21996984-a98b-45ac-b554-05c77f4cc12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:39.381006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-21996984-a98b-45ac-b554-05c77f4cc12b None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:39.382571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-21996984-a98b-45ac-b554-05c77f4cc12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:39.391510 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1125/2259] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:45:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:39.418164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f8b19-71f3-4a30-ba4e-9b44cc5bd3d6 service glance] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:39.438226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f8b19-71f3-4a30-ba4e-9b44cc5bd3d6 service glance] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:39.453756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e91ec3-5e0e-4684-ac7f-24d7ade704d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:39.454430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e91ec3-5e0e-4684-ac7f-24d7ade704d7 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:39.454758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e91ec3-5e0e-4684-ac7f-24d7ade704d7 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:39.455040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e91ec3-5e0e-4684-ac7f-24d7ade704d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:39.467092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e91ec3-5e0e-4684-ac7f-24d7ade704d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:39.469123 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e91ec3-5e0e-4684-ac7f-24d7ade704d7 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:39.485628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1135/2260] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:45:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:45:49.364450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=23 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:49.479108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=54 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:45:50.356303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c9cd1-426e-4a1a-9173-a9470ac721ff None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:50.356590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c9cd1-426e-4a1a-9173-a9470ac721ff None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:50.356869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c9cd1-426e-4a1a-9173-a9470ac721ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:50.374632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-980c9cd1-426e-4a1a-9173-a9470ac721ff None None] MFA Rules not processed for user `104a9c10adde46828fb8592a095e86df`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:50.375854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-980c9cd1-426e-4a1a-9173-a9470ac721ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:50.383889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1126/2261] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:45:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:53.112050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd285f-290f-4038-9c5b-a5e35de44ac7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:53.112596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd285f-290f-4038-9c5b-a5e35de44ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:53.112819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd285f-290f-4038-9c5b-a5e35de44ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:53.131884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-effd285f-290f-4038-9c5b-a5e35de44ac7 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:53.133439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-effd285f-290f-4038-9c5b-a5e35de44ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:53.142228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1136/2262] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:45:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:53.155086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c9cd1-426e-4a1a-9173-a9470ac721ff None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:53.167823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c9cd1-426e-4a1a-9173-a9470ac721ff None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:53.173250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91902d68-0ae8-4832-bfda-e9908f10c981 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:53.173807 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91902d68-0ae8-4832-bfda-e9908f10c981 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:53.174284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91902d68-0ae8-4832-bfda-e9908f10c981 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:53.174728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91902d68-0ae8-4832-bfda-e9908f10c981 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:53.185331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91902d68-0ae8-4832-bfda-e9908f10c981 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:53.187167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91902d68-0ae8-4832-bfda-e9908f10c981 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:53.200032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1127/2263] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:45:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:45:53.963693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512be61-b78c-4d65-8e62-aba55595e11d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:53.963987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512be61-b78c-4d65-8e62-aba55595e11d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:53.964284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512be61-b78c-4d65-8e62-aba55595e11d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:53.966177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1137/2264] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:45:53.981420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163de99a-75c0-42f1-9b01-f9c64ab844b0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:53.981911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163de99a-75c0-42f1-9b01-f9c64ab844b0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:53.982294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163de99a-75c0-42f1-9b01-f9c64ab844b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:54.016158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-163de99a-75c0-42f1-9b01-f9c64ab844b0 None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:54.021020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-163de99a-75c0-42f1-9b01-f9c64ab844b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:54.116822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1128/2265] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:45:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:54.128992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512be61-b78c-4d65-8e62-aba55595e11d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:54.130889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0512be61-b78c-4d65-8e62-aba55595e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:54.145905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512be61-b78c-4d65-8e62-aba55595e11d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:54.149803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d08e9-b0c2-4395-a30e-06ae7686bf89 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc21faf2237d405bba03e5309d93629f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:54.150740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d08e9-b0c2-4395-a30e-06ae7686bf89 service glance] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:54.151949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d08e9-b0c2-4395-a30e-06ae7686bf89 service glance] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:54.152438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d08e9-b0c2-4395-a30e-06ae7686bf89 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:54.168276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d08e9-b0c2-4395-a30e-06ae7686bf89 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:54.169821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d08e9-b0c2-4395-a30e-06ae7686bf89 service glance] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:54.181739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1138/2266] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:45:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:45:54.257107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee206d47-941d-4f8a-b0bd-72c58c453026 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:54.257740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee206d47-941d-4f8a-b0bd-72c58c453026 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:54.257740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee206d47-941d-4f8a-b0bd-72c58c453026 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:54.259145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1129/2267] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:45:54.264267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee39a0-f14d-46e7-9e82-43561d679ac6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:54.264645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee39a0-f14d-46e7-9e82-43561d679ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:54.265020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee39a0-f14d-46e7-9e82-43561d679ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:54.283214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d3ee39a0-f14d-46e7-9e82-43561d679ac6 None None] MFA Rules not processed for user `c57d44c19d15496e96c6e37510ce67fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:54.284859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d3ee39a0-f14d-46e7-9e82-43561d679ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:54.339593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1139/2268] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:45:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:54.352279 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee206d47-941d-4f8a-b0bd-72c58c453026 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:54.365174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee206d47-941d-4f8a-b0bd-72c58c453026 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:54.369489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85de51-cd14-4f42-892a-86ae42e175b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:54.370246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85de51-cd14-4f42-892a-86ae42e175b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:54.370598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85de51-cd14-4f42-892a-86ae42e175b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:54.370948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85de51-cd14-4f42-892a-86ae42e175b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:54.379472 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85de51-cd14-4f42-892a-86ae42e175b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:54.381053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85de51-cd14-4f42-892a-86ae42e175b1 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:54.400209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1130/2269] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:45:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:45:54.514121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8803fb-f01e-4e0e-a4b7-771e20ff83e7 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:54.514741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8803fb-f01e-4e0e-a4b7-771e20ff83e7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:54.515096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8803fb-f01e-4e0e-a4b7-771e20ff83e7 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:54.517096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1140/2270] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 13 20:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:45:54.526275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb47e53-8e7c-4fd0-b46a-7ef53a9f1ea2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:54.526562 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb47e53-8e7c-4fd0-b46a-7ef53a9f1ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:54.526847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb47e53-8e7c-4fd0-b46a-7ef53a9f1ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:54.546968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bcb47e53-8e7c-4fd0-b46a-7ef53a9f1ea2 None None] MFA Rules not processed for user `e80bb5b10ce945cabbfb88ca5df92e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:45:54.548211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bcb47e53-8e7c-4fd0-b46a-7ef53a9f1ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:45:54.559000 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1131/2271] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 13 20:45:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:45:54.570310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8803fb-f01e-4e0e-a4b7-771e20ff83e7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:45:54.584062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8803fb-f01e-4e0e-a4b7-771e20ff83e7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:45:54.588273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc3564-42c9-43ae-bed0-1abd5407022a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:45:54.588949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc3564-42c9-43ae-bed0-1abd5407022a service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:45:54.589242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc3564-42c9-43ae-bed0-1abd5407022a service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:45:54.589523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc3564-42c9-43ae-bed0-1abd5407022a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:45:54.600525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc3564-42c9-43ae-bed0-1abd5407022a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:45:54.601982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc3564-42c9-43ae-bed0-1abd5407022a service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:45:54.613003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1141/2272] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:45:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:46:04.541126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=49 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:46:04.608264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=38 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:46:14.545685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991a26b-a12f-49ae-8f25-7698e32a27bd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:14.546244 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991a26b-a12f-49ae-8f25-7698e32a27bd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:14.546244 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991a26b-a12f-49ae-8f25-7698e32a27bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:14.565561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e991a26b-a12f-49ae-8f25-7698e32a27bd None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:46:14.566755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e991a26b-a12f-49ae-8f25-7698e32a27bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:46:14.575451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1132/2273] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:46:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:46:14.587258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc3564-42c9-43ae-bed0-1abd5407022a service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:46:14.598784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc3564-42c9-43ae-bed0-1abd5407022a service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:46:14.604422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b2224-b589-4d01-9ac6-031e19f7f39b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:46:14.604748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b2224-b589-4d01-9ac6-031e19f7f39b service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:14.605036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b2224-b589-4d01-9ac6-031e19f7f39b service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:14.605314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b2224-b589-4d01-9ac6-031e19f7f39b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:14.613670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b2224-b589-4d01-9ac6-031e19f7f39b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:46:14.615404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b2224-b589-4d01-9ac6-031e19f7f39b service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:46:14.623773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1142/2274] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:46:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:46:24.559763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:46:24.620694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:46:24.747913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991a26b-a12f-49ae-8f25-7698e32a27bd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:46:24.758173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991a26b-a12f-49ae-8f25-7698e32a27bd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:46:24.766181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf62018-4074-44bf-b123-2d538414ad02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:46:24.767133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf62018-4074-44bf-b123-2d538414ad02 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:24.767560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf62018-4074-44bf-b123-2d538414ad02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:24.767927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf62018-4074-44bf-b123-2d538414ad02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:24.770046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aaf62018-4074-44bf-b123-2d538414ad02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:46:24.789221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf62018-4074-44bf-b123-2d538414ad02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:46:24.790677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf62018-4074-44bf-b123-2d538414ad02 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:46:24.802002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1133/2275] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:46:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:46:25.379926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b2224-b589-4d01-9ac6-031e19f7f39b service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:46:25.391473 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b2224-b589-4d01-9ac6-031e19f7f39b service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:46:25.395968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d9c5-e11f-4bba-8bac-409cc2bc8116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:46:25.396615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d9c5-e11f-4bba-8bac-409cc2bc8116 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:25.396905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d9c5-e11f-4bba-8bac-409cc2bc8116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:25.397188 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d9c5-e11f-4bba-8bac-409cc2bc8116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:25.399422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5531d9c5-e11f-4bba-8bac-409cc2bc8116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:46:25.413549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531d9c5-e11f-4bba-8bac-409cc2bc8116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:46:25.415003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531d9c5-e11f-4bba-8bac-409cc2bc8116 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:46:25.426802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1143/2276] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:46:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:46:34.796948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:46:35.421814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:46:44.481714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b1ee-a9d3-4642-8520-690d7e2400c2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:44.482548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b1ee-a9d3-4642-8520-690d7e2400c2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:44.482548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b1ee-a9d3-4642-8520-690d7e2400c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:44.503867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2234b1ee-a9d3-4642-8520-690d7e2400c2 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:46:44.505365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2234b1ee-a9d3-4642-8520-690d7e2400c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:46:44.514465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1134/2277] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 13 20:46:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:46:44.540331 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d9c5-e11f-4bba-8bac-409cc2bc8116 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:46:44.552048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d9c5-e11f-4bba-8bac-409cc2bc8116 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:46:44.558984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4137cfb-4d5a-484c-9708-10f8d6e88617 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:46:44.559916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4137cfb-4d5a-484c-9708-10f8d6e88617 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:44.560475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4137cfb-4d5a-484c-9708-10f8d6e88617 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:44.561763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4137cfb-4d5a-484c-9708-10f8d6e88617 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:44.573748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4137cfb-4d5a-484c-9708-10f8d6e88617 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:46:44.575862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4137cfb-4d5a-484c-9708-10f8d6e88617 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:46:44.586466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1144/2278] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:46:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:46:47.270271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234b1ee-a9d3-4642-8520-690d7e2400c2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:46:47.281011 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234b1ee-a9d3-4642-8520-690d7e2400c2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:46:47.285638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190a80b-ffc6-44a8-b527-a33ecc69ee26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:46:47.286293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a80b-ffc6-44a8-b527-a33ecc69ee26 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:47.286628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a80b-ffc6-44a8-b527-a33ecc69ee26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:47.286919 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a80b-ffc6-44a8-b527-a33ecc69ee26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:47.296595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190a80b-ffc6-44a8-b527-a33ecc69ee26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:46:47.298101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190a80b-ffc6-44a8-b527-a33ecc69ee26 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:46:47.307730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1135/2279] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:46:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:46:52.004887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4137cfb-4d5a-484c-9708-10f8d6e88617 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:46:52.006474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d4137cfb-4d5a-484c-9708-10f8d6e88617 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:46:52.019488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4137cfb-4d5a-484c-9708-10f8d6e88617 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:46:52.023630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75c549-1a0d-4e44-8efa-c336caf148e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:46:52.024278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75c549-1a0d-4e44-8efa-c336caf148e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:52.024565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75c549-1a0d-4e44-8efa-c336caf148e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:52.024852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75c549-1a0d-4e44-8efa-c336caf148e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:52.027059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-be75c549-1a0d-4e44-8efa-c336caf148e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:46:52.037507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75c549-1a0d-4e44-8efa-c336caf148e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:46:52.039018 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75c549-1a0d-4e44-8efa-c336caf148e5 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:46:52.047438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1145/2280] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:46:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:46:54.480320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190a80b-ffc6-44a8-b527-a33ecc69ee26 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:46:54.482950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6190a80b-ffc6-44a8-b527-a33ecc69ee26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:46:54.500752 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190a80b-ffc6-44a8-b527-a33ecc69ee26 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:46:54.507862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:46:54.508504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:54.508816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:54.509096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:54.511238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:46:54.571406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:46:54.573053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:46:54.582193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1136/2281] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:46:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:46:54.765507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57562fc-fb09-4865-943a-f6baba2d88fe None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:46:54.766028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57562fc-fb09-4865-943a-f6baba2d88fe None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:46:54.766028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57562fc-fb09-4865-943a-f6baba2d88fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:46:54.784728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d57562fc-fb09-4865-943a-f6baba2d88fe None None] MFA Rules not processed for user `798de1406eac40b6bc5cf2f5bea4846f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:46:54.785938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d57562fc-fb09-4865-943a-f6baba2d88fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:46:54.793717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1146/2282] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:46:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:47:04.116068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:47:04.117739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:04.132920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c94bd-e79e-449c-85fe-da8f8395b7b9 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:47:04.140120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763ad1e-7f51-4411-8f77-4712ff34e307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:47:04.140906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763ad1e-7f51-4411-8f77-4712ff34e307 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:47:04.141188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763ad1e-7f51-4411-8f77-4712ff34e307 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:47:04.141471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763ad1e-7f51-4411-8f77-4712ff34e307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:47:04.143942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1763ad1e-7f51-4411-8f77-4712ff34e307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:04.214236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763ad1e-7f51-4411-8f77-4712ff34e307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:47:04.215900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763ad1e-7f51-4411-8f77-4712ff34e307 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:47:04.228647 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1137/2283] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:47:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:47:04.779804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=15 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:47:06.503127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57562fc-fb09-4865-943a-f6baba2d88fe None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:47:06.522427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57562fc-fb09-4865-943a-f6baba2d88fe None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:47:06.529448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dba61-0c31-4be5-8298-cda9e8615970 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:47:06.530122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dba61-0c31-4be5-8298-cda9e8615970 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:47:06.530399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dba61-0c31-4be5-8298-cda9e8615970 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:47:06.530671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dba61-0c31-4be5-8298-cda9e8615970 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:47:06.532795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c90dba61-0c31-4be5-8298-cda9e8615970 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:06.593508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dba61-0c31-4be5-8298-cda9e8615970 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:47:06.595247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dba61-0c31-4be5-8298-cda9e8615970 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:47:06.605497 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1147/2284] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:47:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:47:10.153085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763ad1e-7f51-4411-8f77-4712ff34e307 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:47:10.159256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=66 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:47:10.167105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763ad1e-7f51-4411-8f77-4712ff34e307 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:47:10.174520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f0d52-cff7-4789-9b3d-0dbb4e9cf288 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:47:10.175290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f0d52-cff7-4789-9b3d-0dbb4e9cf288 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:47:10.175680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f0d52-cff7-4789-9b3d-0dbb4e9cf288 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:47:10.176129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f0d52-cff7-4789-9b3d-0dbb4e9cf288 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:47:10.178176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-279f0d52-cff7-4789-9b3d-0dbb4e9cf288 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:10.194077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f0d52-cff7-4789-9b3d-0dbb4e9cf288 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:47:10.196036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f0d52-cff7-4789-9b3d-0dbb4e9cf288 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:47:10.208519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1138/2285] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:47:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:47:16.600572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=28 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:47:20.202501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:47:20.325105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c0257-d73a-4949-bc35-645c17ee3b00 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:47:20.325888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c0257-d73a-4949-bc35-645c17ee3b00 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:47:20.326097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c0257-d73a-4949-bc35-645c17ee3b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:47:20.346081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ce3c0257-d73a-4949-bc35-645c17ee3b00 None None] MFA Rules not processed for user `0b22e96941dd414da4fa22ed4bd36984`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:47:20.347305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ce3c0257-d73a-4949-bc35-645c17ee3b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:20.354380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1148/2286] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:47:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:47:20.672241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f0d52-cff7-4789-9b3d-0dbb4e9cf288 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:47:20.681878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f0d52-cff7-4789-9b3d-0dbb4e9cf288 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:47:20.686199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e078ec4-5553-4278-9b40-c6dc69f55f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:47:20.686883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e078ec4-5553-4278-9b40-c6dc69f55f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:47:20.687488 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e078ec4-5553-4278-9b40-c6dc69f55f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:47:20.687796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e078ec4-5553-4278-9b40-c6dc69f55f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:47:20.689881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8e078ec4-5553-4278-9b40-c6dc69f55f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:20.699626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e078ec4-5553-4278-9b40-c6dc69f55f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:47:20.701092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e078ec4-5553-4278-9b40-c6dc69f55f89 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:47:20.709349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1139/2287] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:47:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:47:21.026831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c0257-d73a-4949-bc35-645c17ee3b00 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:47:21.028343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ce3c0257-d73a-4949-bc35-645c17ee3b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:21.109343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c0257-d73a-4949-bc35-645c17ee3b00 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:47:21.114373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe64f7-e7e3-4ad7-95b1-bed1d55caf63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:47:21.115088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe64f7-e7e3-4ad7-95b1-bed1d55caf63 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:47:21.115376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe64f7-e7e3-4ad7-95b1-bed1d55caf63 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:47:21.115692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe64f7-e7e3-4ad7-95b1-bed1d55caf63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:47:21.117474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8efe64f7-e7e3-4ad7-95b1-bed1d55caf63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:21.128524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe64f7-e7e3-4ad7-95b1-bed1d55caf63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:47:21.130149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe64f7-e7e3-4ad7-95b1-bed1d55caf63 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:47:21.140021 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1149/2288] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:47:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:47:30.705508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:47:31.135804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=35 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:47:31.286662 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e078ec4-5553-4278-9b40-c6dc69f55f89 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:47:31.304182 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e078ec4-5553-4278-9b40-c6dc69f55f89 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:47:31.310513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab0946-ebd9-48bf-91a7-f4b28b936309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:47:31.311171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0946-ebd9-48bf-91a7-f4b28b936309 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:47:31.311488 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0946-ebd9-48bf-91a7-f4b28b936309 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:47:31.311771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0946-ebd9-48bf-91a7-f4b28b936309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:47:31.314235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a9ab0946-ebd9-48bf-91a7-f4b28b936309 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:31.327778 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0946-ebd9-48bf-91a7-f4b28b936309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:47:31.329425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0946-ebd9-48bf-91a7-f4b28b936309 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:47:31.341062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1140/2289] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:47:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:47:41.337004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:47:53.051195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe64f7-e7e3-4ad7-95b1-bed1d55caf63 service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:47:53.064821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe64f7-e7e3-4ad7-95b1-bed1d55caf63 service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:47:53.071735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873777d-a82d-4ba0-b7e9-359084c71acf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:47:53.072476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873777d-a82d-4ba0-b7e9-359084c71acf service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:47:53.072760 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873777d-a82d-4ba0-b7e9-359084c71acf service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:47:53.073029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873777d-a82d-4ba0-b7e9-359084c71acf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:47:53.075127 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1873777d-a82d-4ba0-b7e9-359084c71acf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:47:53.136965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873777d-a82d-4ba0-b7e9-359084c71acf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:47:53.138578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873777d-a82d-4ba0-b7e9-359084c71acf service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:47:53.147843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1150/2290] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:47:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:03.143612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=28 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:05.103487 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f26b5-0ab8-4f9e-a055-acaf40b47fbf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:05.104016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f26b5-0ab8-4f9e-a055-acaf40b47fbf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:05.104168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f26b5-0ab8-4f9e-a055-acaf40b47fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:05.131512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-df4f26b5-0ab8-4f9e-a055-acaf40b47fbf None None] MFA Rules not processed for user `c54ec6918ad44a31ba7137abb56b89b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:05.132776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-df4f26b5-0ab8-4f9e-a055-acaf40b47fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:05.141444 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1141/2291] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:15.122927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:18.060613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873777d-a82d-4ba0-b7e9-359084c71acf service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:18.075171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873777d-a82d-4ba0-b7e9-359084c71acf service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:18.088168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8fbd8-c2bb-472b-8991-51274d19f350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:18.092049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8fbd8-c2bb-472b-8991-51274d19f350 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:18.092819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8fbd8-c2bb-472b-8991-51274d19f350 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:18.093285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8fbd8-c2bb-472b-8991-51274d19f350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:18.098975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-24f8fbd8-c2bb-472b-8991-51274d19f350 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:18.113724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8fbd8-c2bb-472b-8991-51274d19f350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:18.116510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8fbd8-c2bb-472b-8991-51274d19f350 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:18.128963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1151/2292] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:28.124450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:30.214039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f26b5-0ab8-4f9e-a055-acaf40b47fbf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:30.224368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f26b5-0ab8-4f9e-a055-acaf40b47fbf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:30.228143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:30.228793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:30.229069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:30.229349 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:30.231254 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:30.302753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:30.304236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:30.315246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1142/2293] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:33.410433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8fbd8-c2bb-472b-8991-51274d19f350 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:33.421793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8fbd8-c2bb-472b-8991-51274d19f350 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:33.427019 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b62e2d-b20f-4756-b6d5-adfd26343b4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:33.427711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b62e2d-b20f-4756-b6d5-adfd26343b4d service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:33.428022 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b62e2d-b20f-4756-b6d5-adfd26343b4d service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:33.428284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b62e2d-b20f-4756-b6d5-adfd26343b4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:33.430045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d9b62e2d-b20f-4756-b6d5-adfd26343b4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:33.441929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b62e2d-b20f-4756-b6d5-adfd26343b4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:33.443548 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b62e2d-b20f-4756-b6d5-adfd26343b4d service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:33.454180 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1152/2294] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:33.472322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:33.473880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:33.487213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1a0c6-0c64-4b69-8d6b-fe4d1fadaefa service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:33.492805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:33.493927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:33.494236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:33.494485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] PATH_INFO: `/v3/users/798de1406eac40b6bc5cf2f5bea4846f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:33.495872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=798de1406eac40b6bc5cf2f5bea4846f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:33.497857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:33.550545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '798de1406eac40b6bc5cf2f5bea4846f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.551141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '798de1406eac40b6bc5cf2f5bea4846f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.563305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '798de1406eac40b6bc5cf2f5bea4846f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.570613 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '798de1406eac40b6bc5cf2f5bea4846f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.571408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-57a90fc8-78b3-499d-83d4-7e81fd249b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '798de1406eac40b6bc5cf2f5bea4846f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.585989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1143/2295] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:48:33 2022] DELETE /identity/v3/users/798de1406eac40b6bc5cf2f5bea4846f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:33.773396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b62e2d-b20f-4756-b6d5-adfd26343b4d service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:33.785951 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b62e2d-b20f-4756-b6d5-adfd26343b4d service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:33.843851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:33.844733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:33.845018 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:33.845292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] PATH_INFO: `/v3/projects/670dd6acab474449a679024e1d357424` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:33.846476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=670dd6acab474449a679024e1d357424)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:33.848467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:33.896236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670dd6acab474449a679024e1d357424', 'request_id': 'req-613f6251-cd14-4200-9ba3-e0ec15c65e9f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.904953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670dd6acab474449a679024e1d357424', 'request_id': 'req-613f6251-cd14-4200-9ba3-e0ec15c65e9f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.911143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670dd6acab474449a679024e1d357424', 'request_id': 'req-613f6251-cd14-4200-9ba3-e0ec15c65e9f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.923137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] The token cache is being invalidate because project 670dd6acab474449a679024e1d357424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:48:33.923427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.924102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-613f6251-cd14-4200-9ba3-e0ec15c65e9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:33.925672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1153/2296] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:48:33 2022] DELETE /identity/v3/projects/670dd6acab474449a679024e1d357424 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:33.936763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b89d3-d321-4935-834f-659ee1d06dc2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:33.937028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b89d3-d321-4935-834f-659ee1d06dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:33.937312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b89d3-d321-4935-834f-659ee1d06dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:33.956971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8a0b89d3-d321-4935-834f-659ee1d06dc2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:33.958190 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8a0b89d3-d321-4935-834f-659ee1d06dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:34.023979 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1144/2297] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:34.032789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5ebc1-b2f6-4c9b-aecb-5199910f1da9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.033079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5ebc1-b2f6-4c9b-aecb-5199910f1da9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.033359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5ebc1-b2f6-4c9b-aecb-5199910f1da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.052629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0cb5ebc1-b2f6-4c9b-aecb-5199910f1da9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:34.053863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0cb5ebc1-b2f6-4c9b-aecb-5199910f1da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:34.063072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1154/2298] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:34.066997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b89d3-d321-4935-834f-659ee1d06dc2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.077230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b89d3-d321-4935-834f-659ee1d06dc2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.082138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f17fbd-a576-43fb-8c62-c0d6d5f94068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.082814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17fbd-a576-43fb-8c62-c0d6d5f94068 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.083089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17fbd-a576-43fb-8c62-c0d6d5f94068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.083342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17fbd-a576-43fb-8c62-c0d6d5f94068 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.084100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f17fbd-a576-43fb-8c62-c0d6d5f94068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:48:34.084403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f17fbd-a576-43fb-8c62-c0d6d5f94068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.085999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f17fbd-a576-43fb-8c62-c0d6d5f94068 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.087638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:34.101285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1145/2299] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:48:34.105569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5ebc1-b2f6-4c9b-aecb-5199910f1da9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.115381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5ebc1-b2f6-4c9b-aecb-5199910f1da9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.121020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd7dff-4ac2-433b-a3cc-4522d80f2822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.121681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd7dff-4ac2-433b-a3cc-4522d80f2822 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.121968 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd7dff-4ac2-433b-a3cc-4522d80f2822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.122257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd7dff-4ac2-433b-a3cc-4522d80f2822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.122965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd7dff-4ac2-433b-a3cc-4522d80f2822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.124922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd7dff-4ac2-433b-a3cc-4522d80f2822 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.152099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1155/2300] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:48:34 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:48:34.156411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f17fbd-a576-43fb-8c62-c0d6d5f94068 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.166801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f17fbd-a576-43fb-8c62-c0d6d5f94068 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.232298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a266e9f-9dba-445e-b00e-f577007e66ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.232984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a266e9f-9dba-445e-b00e-f577007e66ed admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.233268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a266e9f-9dba-445e-b00e-f577007e66ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.233545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a266e9f-9dba-445e-b00e-f577007e66ed admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.234241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a266e9f-9dba-445e-b00e-f577007e66ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.236191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a266e9f-9dba-445e-b00e-f577007e66ed admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.260406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1146/2301] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:48:34 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:34.261633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd7dff-4ac2-433b-a3cc-4522d80f2822 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.273099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd7dff-4ac2-433b-a3cc-4522d80f2822 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.278801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39515e21-d128-41c5-b364-c31205e30d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.279567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39515e21-d128-41c5-b364-c31205e30d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.279836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39515e21-d128-41c5-b364-c31205e30d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.280110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39515e21-d128-41c5-b364-c31205e30d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.280874 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39515e21-d128-41c5-b364-c31205e30d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.282659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39515e21-d128-41c5-b364-c31205e30d95 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.293824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1156/2302] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:48:34.298019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a266e9f-9dba-445e-b00e-f577007e66ed admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.308333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a266e9f-9dba-445e-b00e-f577007e66ed admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.314249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba05c0-28cd-4c2b-b15a-07a61027333e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.315643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba05c0-28cd-4c2b-b15a-07a61027333e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.316123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba05c0-28cd-4c2b-b15a-07a61027333e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.316417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba05c0-28cd-4c2b-b15a-07a61027333e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.317229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba05c0-28cd-4c2b-b15a-07a61027333e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.318856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba05c0-28cd-4c2b-b15a-07a61027333e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.330202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1147/2303] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:48:34.334648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39515e21-d128-41c5-b364-c31205e30d95 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.345172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39515e21-d128-41c5-b364-c31205e30d95 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.350100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c010630-f16f-44a2-81cf-7b98bc199c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.350998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c010630-f16f-44a2-81cf-7b98bc199c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.351290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c010630-f16f-44a2-81cf-7b98bc199c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.351630 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c010630-f16f-44a2-81cf-7b98bc199c86 admin admin] PATH_INFO: `/v3/projects/b46bafcc079e4515a9b9bf21ac8711be/users/468216bdacfb4ef4b78f286abe4065f7/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.367146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c010630-f16f-44a2-81cf-7b98bc199c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46bafcc079e4515a9b9bf21ac8711be, user_id=468216bdacfb4ef4b78f286abe4065f7, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.369927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c010630-f16f-44a2-81cf-7b98bc199c86 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.385803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1157/2304] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:48:34 2022] PUT /identity/v3/projects/b46bafcc079e4515a9b9bf21ac8711be/users/468216bdacfb4ef4b78f286abe4065f7/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:34.395868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24964b-cdba-43eb-92d9-90cdc22e5549 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.396168 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24964b-cdba-43eb-92d9-90cdc22e5549 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.396456 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24964b-cdba-43eb-92d9-90cdc22e5549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.423529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4a24964b-cdba-43eb-92d9-90cdc22e5549 None None] MFA Rules not processed for user `468216bdacfb4ef4b78f286abe4065f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:34.424695 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4a24964b-cdba-43eb-92d9-90cdc22e5549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:34.491815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1148/2305] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:34.496459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c010630-f16f-44a2-81cf-7b98bc199c86 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.510471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:34.559897 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c010630-f16f-44a2-81cf-7b98bc199c86 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.565033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04af37-3ff9-428c-ab8d-e92ed8e0a62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.565688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04af37-3ff9-428c-ab8d-e92ed8e0a62b admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.565945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04af37-3ff9-428c-ab8d-e92ed8e0a62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.566234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04af37-3ff9-428c-ab8d-e92ed8e0a62b admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.566908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04af37-3ff9-428c-ab8d-e92ed8e0a62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.568857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04af37-3ff9-428c-ab8d-e92ed8e0a62b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.596768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1158/2306] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:48:34 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:34.602452 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24964b-cdba-43eb-92d9-90cdc22e5549 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.614097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24964b-cdba-43eb-92d9-90cdc22e5549 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.676116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df11cc3-fcb6-4199-a62b-7f649492085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.676812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df11cc3-fcb6-4199-a62b-7f649492085c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.677136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df11cc3-fcb6-4199-a62b-7f649492085c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.677414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df11cc3-fcb6-4199-a62b-7f649492085c admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.678110 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df11cc3-fcb6-4199-a62b-7f649492085c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.680073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df11cc3-fcb6-4199-a62b-7f649492085c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.688078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:34.700984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1149/2307] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:48:34 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:34.705133 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04af37-3ff9-428c-ab8d-e92ed8e0a62b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.715504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04af37-3ff9-428c-ab8d-e92ed8e0a62b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.721002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019408f7-a06c-4a73-8d75-ce00f4cc86f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.721660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019408f7-a06c-4a73-8d75-ce00f4cc86f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.721944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019408f7-a06c-4a73-8d75-ce00f4cc86f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.722203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019408f7-a06c-4a73-8d75-ce00f4cc86f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.722920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019408f7-a06c-4a73-8d75-ce00f4cc86f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.724555 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019408f7-a06c-4a73-8d75-ce00f4cc86f2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.735737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1159/2308] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:48:34.740049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df11cc3-fcb6-4199-a62b-7f649492085c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.750927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df11cc3-fcb6-4199-a62b-7f649492085c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.756416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f224473-36e6-4b1b-9cf5-7cf8347dd0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.757303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f224473-36e6-4b1b-9cf5-7cf8347dd0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.757589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f224473-36e6-4b1b-9cf5-7cf8347dd0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.757869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f224473-36e6-4b1b-9cf5-7cf8347dd0e9 admin admin] PATH_INFO: `/v3/projects/df38dac1243d4437bf8ec860d0dbcb15/users/baa663c78c104dd2826189e652a88405/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.774327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f224473-36e6-4b1b-9cf5-7cf8347dd0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df38dac1243d4437bf8ec860d0dbcb15, user_id=baa663c78c104dd2826189e652a88405, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.777037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f224473-36e6-4b1b-9cf5-7cf8347dd0e9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.793609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1150/2309] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:48:34 2022] PUT /identity/v3/projects/df38dac1243d4437bf8ec860d0dbcb15/users/baa663c78c104dd2826189e652a88405/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:34.797585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019408f7-a06c-4a73-8d75-ce00f4cc86f2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.807435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019408f7-a06c-4a73-8d75-ce00f4cc86f2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.873925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93093342-177e-4d16-941a-5d408dee1057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.874511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093342-177e-4d16-941a-5d408dee1057 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.874801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093342-177e-4d16-941a-5d408dee1057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.875079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093342-177e-4d16-941a-5d408dee1057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.875842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93093342-177e-4d16-941a-5d408dee1057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.877451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93093342-177e-4d16-941a-5d408dee1057 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.887374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1160/2310] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:48:34.891960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f224473-36e6-4b1b-9cf5-7cf8347dd0e9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:34.902899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f224473-36e6-4b1b-9cf5-7cf8347dd0e9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:34.908974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05832f09-83f8-4df7-b325-9b1644df3681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:34.910138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05832f09-83f8-4df7-b325-9b1644df3681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.910447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05832f09-83f8-4df7-b325-9b1644df3681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.910730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05832f09-83f8-4df7-b325-9b1644df3681 admin admin] PATH_INFO: `/v3/projects/df38dac1243d4437bf8ec860d0dbcb15/users/baa663c78c104dd2826189e652a88405/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.913584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05832f09-83f8-4df7-b325-9b1644df3681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df38dac1243d4437bf8ec860d0dbcb15, user_id=baa663c78c104dd2826189e652a88405, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:34.916724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05832f09-83f8-4df7-b325-9b1644df3681 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:34.929112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1151/2311] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:48:34 2022] PUT /identity/v3/projects/df38dac1243d4437bf8ec860d0dbcb15/users/baa663c78c104dd2826189e652a88405/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:34.939864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed641410-ba3e-4db5-b818-7aedab473f80 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:34.940167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed641410-ba3e-4db5-b818-7aedab473f80 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:34.940422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed641410-ba3e-4db5-b818-7aedab473f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:34.966839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ed641410-ba3e-4db5-b818-7aedab473f80 None None] MFA Rules not processed for user `baa663c78c104dd2826189e652a88405`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:34.968062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ed641410-ba3e-4db5-b818-7aedab473f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:35.038140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1161/2312] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:34 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:35.049614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05832f09-83f8-4df7-b325-9b1644df3681 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:35.051655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-05832f09-83f8-4df7-b325-9b1644df3681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:35.111826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05832f09-83f8-4df7-b325-9b1644df3681 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:35.116035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbace1c2-57e9-415c-b919-83836d8e8f27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:35.116693 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbace1c2-57e9-415c-b919-83836d8e8f27 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:35.117005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbace1c2-57e9-415c-b919-83836d8e8f27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:35.117290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbace1c2-57e9-415c-b919-83836d8e8f27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:35.126704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbace1c2-57e9-415c-b919-83836d8e8f27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:35.128135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbace1c2-57e9-415c-b919-83836d8e8f27 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:35.137731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1152/2313] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:35 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:35.212171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed641410-ba3e-4db5-b818-7aedab473f80 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:35.222927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed641410-ba3e-4db5-b818-7aedab473f80 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:35.227243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd52155-d593-4e34-89bb-b560eccf59bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:35.227940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd52155-d593-4e34-89bb-b560eccf59bb service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:35.228198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd52155-d593-4e34-89bb-b560eccf59bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:35.228475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd52155-d593-4e34-89bb-b560eccf59bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:35.234056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:35.292985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd52155-d593-4e34-89bb-b560eccf59bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:35.294498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd52155-d593-4e34-89bb-b560eccf59bb service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:35.304994 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1162/2314] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:44.726920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80977313-1248-4980-9015-449264a3d3d9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:44.727591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80977313-1248-4980-9015-449264a3d3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:44.727713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80977313-1248-4980-9015-449264a3d3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:44.746581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-80977313-1248-4980-9015-449264a3d3d9 None None] MFA Rules not processed for user `bf4d42cf5bec422b96f421b98c84491d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:44.747967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-80977313-1248-4980-9015-449264a3d3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:44.810169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1153/2315] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:45.116148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd52155-d593-4e34-89bb-b560eccf59bb service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:45.117692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fcd52155-d593-4e34-89bb-b560eccf59bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:45.178721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd52155-d593-4e34-89bb-b560eccf59bb service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:45.183013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557790fe-6e59-4322-958e-85e46f029dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:45.183702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557790fe-6e59-4322-958e-85e46f029dde service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:45.184029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557790fe-6e59-4322-958e-85e46f029dde service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:45.184308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557790fe-6e59-4322-958e-85e46f029dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:45.186873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-557790fe-6e59-4322-958e-85e46f029dde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:45.196893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557790fe-6e59-4322-958e-85e46f029dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:45.198356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557790fe-6e59-4322-958e-85e46f029dde service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:45.206367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1163/2316] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:46.894341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80977313-1248-4980-9015-449264a3d3d9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:46.978401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80977313-1248-4980-9015-449264a3d3d9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:46.985128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:46.986802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:46.987088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:46.987365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] PATH_INFO: `/v3/users/468216bdacfb4ef4b78f286abe4065f7` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:46.989020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=468216bdacfb4ef4b78f286abe4065f7)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:46.991365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:46.993786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:47.045078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468216bdacfb4ef4b78f286abe4065f7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.045949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468216bdacfb4ef4b78f286abe4065f7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.059442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468216bdacfb4ef4b78f286abe4065f7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.068047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468216bdacfb4ef4b78f286abe4065f7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.068677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468216bdacfb4ef4b78f286abe4065f7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.083691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1154/2317] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:48:46 2022] DELETE /identity/v3/users/468216bdacfb4ef4b78f286abe4065f7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:47.088324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d34e03-0bf0-44ad-9161-43dd323ae8ba None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:47.088786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d34e03-0bf0-44ad-9161-43dd323ae8ba None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:47.088922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d34e03-0bf0-44ad-9161-43dd323ae8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:47.106577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a4d34e03-0bf0-44ad-9161-43dd323ae8ba None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:47.107822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a4d34e03-0bf0-44ad-9161-43dd323ae8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:47.168588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1164/2318] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:47.182134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:47.183814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:47.245426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd97cd-869d-43f1-902f-041e851839d5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:47.249702 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb778f-e168-456b-9a70-d8f853ccb2bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:47.250420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb778f-e168-456b-9a70-d8f853ccb2bf service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:47.250741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb778f-e168-456b-9a70-d8f853ccb2bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:47.251026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb778f-e168-456b-9a70-d8f853ccb2bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:47.261140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb778f-e168-456b-9a70-d8f853ccb2bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:47.262621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb778f-e168-456b-9a70-d8f853ccb2bf service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:47.272573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1155/2319] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:47.689822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d34e03-0bf0-44ad-9161-43dd323ae8ba None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:47.691227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a4d34e03-0bf0-44ad-9161-43dd323ae8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:47.704121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d34e03-0bf0-44ad-9161-43dd323ae8ba None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:47.715532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:47.716815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:47.717247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:47.718418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] PATH_INFO: `/v3/projects/b46bafcc079e4515a9b9bf21ac8711be` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:47.720117 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b46bafcc079e4515a9b9bf21ac8711be)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:47.722090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:47.790750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:47.794763 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b46bafcc079e4515a9b9bf21ac8711be', 'request_id': 'req-1affb805-f04d-4ba9-848b-b376b457377d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.805110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b46bafcc079e4515a9b9bf21ac8711be', 'request_id': 'req-1affb805-f04d-4ba9-848b-b376b457377d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.813576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b46bafcc079e4515a9b9bf21ac8711be', 'request_id': 'req-1affb805-f04d-4ba9-848b-b376b457377d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.826474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] The token cache is being invalidate because project b46bafcc079e4515a9b9bf21ac8711be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:48:47.827111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.828035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.830098 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1165/2320] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:48:47 2022] DELETE /identity/v3/projects/b46bafcc079e4515a9b9bf21ac8711be => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:47.833967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb778f-e168-456b-9a70-d8f853ccb2bf service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:47.844428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb778f-e168-456b-9a70-d8f853ccb2bf service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:47.902901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:47.903828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:47.904117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:47.904398 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] PATH_INFO: `/v3/users/baa663c78c104dd2826189e652a88405` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:47.905594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa663c78c104dd2826189e652a88405)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:47.907560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:47.937201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:47.959824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa663c78c104dd2826189e652a88405'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.960470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa663c78c104dd2826189e652a88405'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.972224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa663c78c104dd2826189e652a88405'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.980637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa663c78c104dd2826189e652a88405'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.981426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6b440306-7e52-4ea1-9006-7357c43405ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa663c78c104dd2826189e652a88405'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:47.996319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1156/2321] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:48:47 2022] DELETE /identity/v3/users/baa663c78c104dd2826189e652a88405 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:48.292154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:48.293799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:48.307777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affb805-f04d-4ba9-848b-b376b457377d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:48.367535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:48.368452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.368742 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.369025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] PATH_INFO: `/v3/projects/df38dac1243d4437bf8ec860d0dbcb15` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:48.370312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=df38dac1243d4437bf8ec860d0dbcb15)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:48.372276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:48.422989 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df38dac1243d4437bf8ec860d0dbcb15', 'request_id': 'req-4850da83-c1b0-4e5e-af3a-424d082a7fdd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:48.433478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df38dac1243d4437bf8ec860d0dbcb15', 'request_id': 'req-4850da83-c1b0-4e5e-af3a-424d082a7fdd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:48.441090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df38dac1243d4437bf8ec860d0dbcb15', 'request_id': 'req-4850da83-c1b0-4e5e-af3a-424d082a7fdd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:48.452887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] The token cache is being invalidate because project df38dac1243d4437bf8ec860d0dbcb15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:48:48.453323 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:48.454059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4850da83-c1b0-4e5e-af3a-424d082a7fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:48.455728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1166/2322] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:48:48 2022] DELETE /identity/v3/projects/df38dac1243d4437bf8ec860d0dbcb15 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:48.461018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097684f-7603-4949-bbed-cafd255ebb53 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.461312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097684f-7603-4949-bbed-cafd255ebb53 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.461644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097684f-7603-4949-bbed-cafd255ebb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:48.481481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4097684f-7603-4949-bbed-cafd255ebb53 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:48.482736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4097684f-7603-4949-bbed-cafd255ebb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:48.558637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1157/2323] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:48.561555 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083a01d-11a7-4917-b7f4-b1ae81698aa3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.561555 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083a01d-11a7-4917-b7f4-b1ae81698aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.561821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083a01d-11a7-4917-b7f4-b1ae81698aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:48.583532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4083a01d-11a7-4917-b7f4-b1ae81698aa3 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:48.585020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4083a01d-11a7-4917-b7f4-b1ae81698aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:48.592773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1167/2324] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:48.597210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097684f-7603-4949-bbed-cafd255ebb53 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:48.607033 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097684f-7603-4949-bbed-cafd255ebb53 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:48.611823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb44f0d-e52c-4fc3-91b9-a372d2dec9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:48.612501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb44f0d-e52c-4fc3-91b9-a372d2dec9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.612795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb44f0d-e52c-4fc3-91b9-a372d2dec9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.613072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb44f0d-e52c-4fc3-91b9-a372d2dec9d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:48.614351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb44f0d-e52c-4fc3-91b9-a372d2dec9d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:48:48.614351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb44f0d-e52c-4fc3-91b9-a372d2dec9d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:48.615991 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb44f0d-e52c-4fc3-91b9-a372d2dec9d2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:48.644348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1158/2325] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:48:48.648479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083a01d-11a7-4917-b7f4-b1ae81698aa3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:48.660372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083a01d-11a7-4917-b7f4-b1ae81698aa3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:48.665533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049d10b-b8a0-4ad0-bafd-de50b6d32418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:48.666213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049d10b-b8a0-4ad0-bafd-de50b6d32418 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.666528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049d10b-b8a0-4ad0-bafd-de50b6d32418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.666785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049d10b-b8a0-4ad0-bafd-de50b6d32418 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:48.667541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049d10b-b8a0-4ad0-bafd-de50b6d32418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:48.669501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049d10b-b8a0-4ad0-bafd-de50b6d32418 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:48.726904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1168/2326] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:48:48 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:48.731083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb44f0d-e52c-4fc3-91b9-a372d2dec9d2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:48.742387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb44f0d-e52c-4fc3-91b9-a372d2dec9d2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:48.816422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff157f-d606-400b-8e9a-4c0a092b6c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:48.817125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff157f-d606-400b-8e9a-4c0a092b6c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.817409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff157f-d606-400b-8e9a-4c0a092b6c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.817688 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff157f-d606-400b-8e9a-4c0a092b6c88 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:48.818432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff157f-d606-400b-8e9a-4c0a092b6c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:48.820474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff157f-d606-400b-8e9a-4c0a092b6c88 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:48.851750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1159/2327] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:48:48 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:48.852755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049d10b-b8a0-4ad0-bafd-de50b6d32418 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:48.863799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049d10b-b8a0-4ad0-bafd-de50b6d32418 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:48.868616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd328315-4090-4b62-b0d4-ad68f19b77ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:48.869300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd328315-4090-4b62-b0d4-ad68f19b77ac admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.869583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd328315-4090-4b62-b0d4-ad68f19b77ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.869858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd328315-4090-4b62-b0d4-ad68f19b77ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:48.870588 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd328315-4090-4b62-b0d4-ad68f19b77ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:48.872268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd328315-4090-4b62-b0d4-ad68f19b77ac admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:48.883167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1169/2328] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:48:48.887223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff157f-d606-400b-8e9a-4c0a092b6c88 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:48.898196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff157f-d606-400b-8e9a-4c0a092b6c88 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:48.904030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cf0e9-960c-44ec-a204-19acb9c82a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:48.906314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cf0e9-960c-44ec-a204-19acb9c82a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.906659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cf0e9-960c-44ec-a204-19acb9c82a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.906948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cf0e9-960c-44ec-a204-19acb9c82a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:48.908300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cf0e9-960c-44ec-a204-19acb9c82a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:48.910558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cf0e9-960c-44ec-a204-19acb9c82a8e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:48.923099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1160/2329] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:48:48.926876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd328315-4090-4b62-b0d4-ad68f19b77ac admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:48.937810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd328315-4090-4b62-b0d4-ad68f19b77ac admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:48.942642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03192d90-a9a9-4daf-9349-fac68bf420df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:48.943636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192d90-a9a9-4daf-9349-fac68bf420df admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.943984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192d90-a9a9-4daf-9349-fac68bf420df admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.944322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192d90-a9a9-4daf-9349-fac68bf420df admin admin] PATH_INFO: `/v3/projects/1b54d5f8d9014a1792913d8fe16a9c7c/users/954fbc6014444ccf84f4add1a013956d/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:48.958891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03192d90-a9a9-4daf-9349-fac68bf420df admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b54d5f8d9014a1792913d8fe16a9c7c, user_id=954fbc6014444ccf84f4add1a013956d, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:48.961747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03192d90-a9a9-4daf-9349-fac68bf420df admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:48.972216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:48.978185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1170/2330] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:48:48 2022] PUT /identity/v3/projects/1b54d5f8d9014a1792913d8fe16a9c7c/users/954fbc6014444ccf84f4add1a013956d/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:48.988047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8110e1b-b7b7-4528-b5ab-43c00ba39063 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:48.988343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8110e1b-b7b7-4528-b5ab-43c00ba39063 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:48.988654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8110e1b-b7b7-4528-b5ab-43c00ba39063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:49.007565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:49.018313 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c8110e1b-b7b7-4528-b5ab-43c00ba39063 None None] MFA Rules not processed for user `954fbc6014444ccf84f4add1a013956d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:49.020047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c8110e1b-b7b7-4528-b5ab-43c00ba39063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:49.089768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1161/2331] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:49.094233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03192d90-a9a9-4daf-9349-fac68bf420df admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:49.158054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03192d90-a9a9-4daf-9349-fac68bf420df admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:49.162700 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cf7fc-a80d-4b94-98ee-8a4878af8bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:49.163375 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cf7fc-a80d-4b94-98ee-8a4878af8bad admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:49.163697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cf7fc-a80d-4b94-98ee-8a4878af8bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:49.163977 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cf7fc-a80d-4b94-98ee-8a4878af8bad admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:49.164727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cf7fc-a80d-4b94-98ee-8a4878af8bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:49.166645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cf7fc-a80d-4b94-98ee-8a4878af8bad admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:49.194900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1171/2332] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:48:49 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:49.199122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8110e1b-b7b7-4528-b5ab-43c00ba39063 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:49.209668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8110e1b-b7b7-4528-b5ab-43c00ba39063 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:49.267913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a80d9-5ca4-4d8b-9781-90633171a668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:49.268592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a80d9-5ca4-4d8b-9781-90633171a668 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:49.268885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a80d9-5ca4-4d8b-9781-90633171a668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:49.269170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a80d9-5ca4-4d8b-9781-90633171a668 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:49.269873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a80d9-5ca4-4d8b-9781-90633171a668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:49.271854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a80d9-5ca4-4d8b-9781-90633171a668 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:49.295537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1162/2333] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:48:49 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:49.297034 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cf7fc-a80d-4b94-98ee-8a4878af8bad admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:49.307835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cf7fc-a80d-4b94-98ee-8a4878af8bad admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:49.314146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504a4cd-aa69-4d37-b3e6-f1a7d77211b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:49.314898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504a4cd-aa69-4d37-b3e6-f1a7d77211b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:49.315288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504a4cd-aa69-4d37-b3e6-f1a7d77211b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:49.315652 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504a4cd-aa69-4d37-b3e6-f1a7d77211b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:49.316448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504a4cd-aa69-4d37-b3e6-f1a7d77211b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:49.318175 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504a4cd-aa69-4d37-b3e6-f1a7d77211b8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:49.330103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1172/2334] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:48:49.334696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a80d9-5ca4-4d8b-9781-90633171a668 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:49.346144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a80d9-5ca4-4d8b-9781-90633171a668 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:49.352033 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ec4fc-3ead-410b-96cc-a949a07d1c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:49.352964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ec4fc-3ead-410b-96cc-a949a07d1c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:49.353252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ec4fc-3ead-410b-96cc-a949a07d1c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:49.353530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ec4fc-3ead-410b-96cc-a949a07d1c55 admin admin] PATH_INFO: `/v3/projects/eaa863aca42c49f087579fe8907d6694/users/c72d8e093eed4fecb52e8dc862aee906/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:49.369594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ec4fc-3ead-410b-96cc-a949a07d1c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa863aca42c49f087579fe8907d6694, user_id=c72d8e093eed4fecb52e8dc862aee906, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:49.372295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ec4fc-3ead-410b-96cc-a949a07d1c55 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:49.386760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1163/2335] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:48:49 2022] PUT /identity/v3/projects/eaa863aca42c49f087579fe8907d6694/users/c72d8e093eed4fecb52e8dc862aee906/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:49.390743 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504a4cd-aa69-4d37-b3e6-f1a7d77211b8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:49.401521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504a4cd-aa69-4d37-b3e6-f1a7d77211b8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:49.466674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf2704-9534-4b93-a3ca-ccb8407ef22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:49.467352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf2704-9534-4b93-a3ca-ccb8407ef22a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:49.467681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf2704-9534-4b93-a3ca-ccb8407ef22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:49.467967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf2704-9534-4b93-a3ca-ccb8407ef22a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:49.468698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf2704-9534-4b93-a3ca-ccb8407ef22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:49.470326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf2704-9534-4b93-a3ca-ccb8407ef22a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:49.480348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1173/2336] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:48:49.484770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ec4fc-3ead-410b-96cc-a949a07d1c55 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:49.495830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ec4fc-3ead-410b-96cc-a949a07d1c55 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:49.501474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901f373-59f2-4721-bc94-9e6b1248aac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:49.502371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901f373-59f2-4721-bc94-9e6b1248aac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:49.502654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901f373-59f2-4721-bc94-9e6b1248aac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:49.502932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901f373-59f2-4721-bc94-9e6b1248aac0 admin admin] PATH_INFO: `/v3/projects/eaa863aca42c49f087579fe8907d6694/users/c72d8e093eed4fecb52e8dc862aee906/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:49.505151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901f373-59f2-4721-bc94-9e6b1248aac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa863aca42c49f087579fe8907d6694, user_id=c72d8e093eed4fecb52e8dc862aee906, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:49.507832 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901f373-59f2-4721-bc94-9e6b1248aac0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:49.518988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1164/2337] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:48:49 2022] PUT /identity/v3/projects/eaa863aca42c49f087579fe8907d6694/users/c72d8e093eed4fecb52e8dc862aee906/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:49.529269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0bbdb-5c8c-45fc-88cb-81bda717c458 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:49.529569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0bbdb-5c8c-45fc-88cb-81bda717c458 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:49.529857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0bbdb-5c8c-45fc-88cb-81bda717c458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:49.559657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e7e0bbdb-5c8c-45fc-88cb-81bda717c458 None None] MFA Rules not processed for user `c72d8e093eed4fecb52e8dc862aee906`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:49.561167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e7e0bbdb-5c8c-45fc-88cb-81bda717c458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:49.630553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:49.640776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1174/2338] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:49.660945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901f373-59f2-4721-bc94-9e6b1248aac0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:49.662783 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e901f373-59f2-4721-bc94-9e6b1248aac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:49.725043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901f373-59f2-4721-bc94-9e6b1248aac0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:49.729290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4263e0-671c-4af2-b127-e4a611cd3f16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:49.729958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4263e0-671c-4af2-b127-e4a611cd3f16 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:49.730245 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4263e0-671c-4af2-b127-e4a611cd3f16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:49.730529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4263e0-671c-4af2-b127-e4a611cd3f16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:49.736734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:49.740994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4263e0-671c-4af2-b127-e4a611cd3f16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:49.743337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4263e0-671c-4af2-b127-e4a611cd3f16 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:49.753531 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1165/2339] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:50.080412 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0bbdb-5c8c-45fc-88cb-81bda717c458 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:50.089892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0bbdb-5c8c-45fc-88cb-81bda717c458 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:50.095090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6665f82-2841-43d9-8394-1d7cd6a79e22 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72d8e093eed4fecb52e8dc862aee906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaa863aca42c49f087579fe8907d6694', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:50.095788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6665f82-2841-43d9-8394-1d7cd6a79e22 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:50.096073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6665f82-2841-43d9-8394-1d7cd6a79e22 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:50.096358 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6665f82-2841-43d9-8394-1d7cd6a79e22 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:50.097094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6665f82-2841-43d9-8394-1d7cd6a79e22 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:48:50.097407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6665f82-2841-43d9-8394-1d7cd6a79e22 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:50.099094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6665f82-2841-43d9-8394-1d7cd6a79e22 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:50.111850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1175/2340] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:48:50.115916 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4263e0-671c-4af2-b127-e4a611cd3f16 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:50.125671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4263e0-671c-4af2-b127-e4a611cd3f16 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:50.130916 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a49005-cce2-4cc6-8057-ed0086eb97fc tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72d8e093eed4fecb52e8dc862aee906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaa863aca42c49f087579fe8907d6694', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:50.131607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a49005-cce2-4cc6-8057-ed0086eb97fc tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:50.131898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a49005-cce2-4cc6-8057-ed0086eb97fc tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:50.132212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a49005-cce2-4cc6-8057-ed0086eb97fc tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:50.132911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a49005-cce2-4cc6-8057-ed0086eb97fc tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:50.134802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a49005-cce2-4cc6-8057-ed0086eb97fc tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:50.163471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1166/2341] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:48:50 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:50.167694 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6665f82-2841-43d9-8394-1d7cd6a79e22 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:50.180304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6665f82-2841-43d9-8394-1d7cd6a79e22 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:50.239192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72d8e093eed4fecb52e8dc862aee906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaa863aca42c49f087579fe8907d6694', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:50.239885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:50.240184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:50.240468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:50.241194 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:48:50.241506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:50.243131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:50.254680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1176/2342] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:48:50.291219 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a49005-cce2-4cc6-8057-ed0086eb97fc tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:50.301458 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a49005-cce2-4cc6-8057-ed0086eb97fc tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:50.306733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72d8e093eed4fecb52e8dc862aee906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaa863aca42c49f087579fe8907d6694', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:50.307633 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:50.307921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:50.308194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] PATH_INFO: `/v3/projects/3881cd7d10f6472b9b4bb310f3b28cb0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:50.309404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3881cd7d10f6472b9b4bb310f3b28cb0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:50.311702 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:50.365661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3881cd7d10f6472b9b4bb310f3b28cb0', 'request_id': 'req-c4943148-00d5-44ea-a142-a2fc24f6cc80'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.373047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3881cd7d10f6472b9b4bb310f3b28cb0', 'request_id': 'req-c4943148-00d5-44ea-a142-a2fc24f6cc80'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.379722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3881cd7d10f6472b9b4bb310f3b28cb0', 'request_id': 'req-c4943148-00d5-44ea-a142-a2fc24f6cc80'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.395172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] The token cache is being invalidate because project 3881cd7d10f6472b9b4bb310f3b28cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:48:50.395519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.396164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c4943148-00d5-44ea-a142-a2fc24f6cc80 tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.397730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1167/2343] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:48:50 2022] DELETE /identity/v3/projects/3881cd7d10f6472b9b4bb310f3b28cb0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:50.600484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:50.602461 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:50.668247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab46f8b-10e0-4bde-8cac-cd6371a99eee tempest-VolumeQuotaClassesTest-581322314 tempest-VolumeQuotaClassesTest-581322314-project-admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:50.724539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:50.726092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:50.726492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:50.726966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] PATH_INFO: `/v3/users/954fbc6014444ccf84f4add1a013956d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:50.728947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=954fbc6014444ccf84f4add1a013956d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:50.731772 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:50.763727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:50.785688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '954fbc6014444ccf84f4add1a013956d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.786670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '954fbc6014444ccf84f4add1a013956d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.799862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '954fbc6014444ccf84f4add1a013956d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.806529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '954fbc6014444ccf84f4add1a013956d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.816450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '954fbc6014444ccf84f4add1a013956d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:50.822992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1177/2344] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:48:50 2022] DELETE /identity/v3/users/954fbc6014444ccf84f4add1a013956d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:50.827667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfa6ed-4074-410c-8246-3c3f9898f498 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:50.828022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfa6ed-4074-410c-8246-3c3f9898f498 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:50.828312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfa6ed-4074-410c-8246-3c3f9898f498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:50.847361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-07dfa6ed-4074-410c-8246-3c3f9898f498 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:50.848619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-07dfa6ed-4074-410c-8246-3c3f9898f498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:50.908610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1168/2345] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:48:50.919483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:50.921028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:50.981172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f771b5-a94c-484e-8878-e013c5a5f7e9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:50.987776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a89b80-0066-4e4c-8de8-acdafbf79391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:50.988835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a89b80-0066-4e4c-8de8-acdafbf79391 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:50.989326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a89b80-0066-4e4c-8de8-acdafbf79391 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:50.989871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a89b80-0066-4e4c-8de8-acdafbf79391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:51.002453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a89b80-0066-4e4c-8de8-acdafbf79391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:51.003979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a89b80-0066-4e4c-8de8-acdafbf79391 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:51.014701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1178/2346] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:51.305515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfa6ed-4074-410c-8246-3c3f9898f498 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:51.307081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-07dfa6ed-4074-410c-8246-3c3f9898f498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:51.319642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfa6ed-4074-410c-8246-3c3f9898f498 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:51.324583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:51.325465 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:51.325729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:51.326024 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] PATH_INFO: `/v3/projects/1b54d5f8d9014a1792913d8fe16a9c7c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:51.327249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b54d5f8d9014a1792913d8fe16a9c7c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:51.329190 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:51.376502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b54d5f8d9014a1792913d8fe16a9c7c', 'request_id': 'req-45383680-d5b9-45d2-b66b-b8b610c4ab4f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.377590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:51.385236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b54d5f8d9014a1792913d8fe16a9c7c', 'request_id': 'req-45383680-d5b9-45d2-b66b-b8b610c4ab4f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.392476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b54d5f8d9014a1792913d8fe16a9c7c', 'request_id': 'req-45383680-d5b9-45d2-b66b-b8b610c4ab4f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.406524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] The token cache is being invalidate because project 1b54d5f8d9014a1792913d8fe16a9c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:48:51.406819 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.407449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.409025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1169/2347] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:48:51 2022] DELETE /identity/v3/projects/1b54d5f8d9014a1792913d8fe16a9c7c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:51.413054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a89b80-0066-4e4c-8de8-acdafbf79391 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:51.423978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a89b80-0066-4e4c-8de8-acdafbf79391 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:51.484809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:51.485691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:51.485979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:51.486259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] PATH_INFO: `/v3/users/c72d8e093eed4fecb52e8dc862aee906` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:51.487642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c72d8e093eed4fecb52e8dc862aee906)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:51.489625 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:51.541454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c72d8e093eed4fecb52e8dc862aee906'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.542156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c72d8e093eed4fecb52e8dc862aee906'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.543338 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:51.557116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c72d8e093eed4fecb52e8dc862aee906'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.565314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c72d8e093eed4fecb52e8dc862aee906'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.574929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d151d3cf-583f-4bcf-b722-d9bd4b0149d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c72d8e093eed4fecb52e8dc862aee906'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:51.581107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1179/2348] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:48:51 2022] DELETE /identity/v3/users/c72d8e093eed4fecb52e8dc862aee906 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:51.863415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:51.865474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:51.878661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45383680-d5b9-45d2-b66b-b8b610c4ab4f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:51.949240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:51.950381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:51.950738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:51.951097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] PATH_INFO: `/v3/projects/eaa863aca42c49f087579fe8907d6694` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:51.952444 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa863aca42c49f087579fe8907d6694)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:51.954582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.008172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa863aca42c49f087579fe8907d6694', 'request_id': 'req-1a66a9a9-dc01-4297-99bf-45371a4a248a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:52.014876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa863aca42c49f087579fe8907d6694', 'request_id': 'req-1a66a9a9-dc01-4297-99bf-45371a4a248a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:52.020965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa863aca42c49f087579fe8907d6694', 'request_id': 'req-1a66a9a9-dc01-4297-99bf-45371a4a248a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:52.038923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] The token cache is being invalidate because project eaa863aca42c49f087579fe8907d6694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:48:52.039353 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:52.040169 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1a66a9a9-dc01-4297-99bf-45371a4a248a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:48:52.042222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1170/2349] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:48:51 2022] DELETE /identity/v3/projects/eaa863aca42c49f087579fe8907d6694 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:52.046988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b9133-e999-4e8f-b345-1a16c8bb2e92 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.047499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b9133-e999-4e8f-b345-1a16c8bb2e92 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.047586 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b9133-e999-4e8f-b345-1a16c8bb2e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.072716 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5a4b9133-e999-4e8f-b345-1a16c8bb2e92 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:52.073996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5a4b9133-e999-4e8f-b345-1a16c8bb2e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:52.134446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1180/2350] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:52.142152 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd63d1-9615-44a3-a1bb-0abaa7370afb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.142446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd63d1-9615-44a3-a1bb-0abaa7370afb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.142731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd63d1-9615-44a3-a1bb-0abaa7370afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.161687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a1fd63d1-9615-44a3-a1bb-0abaa7370afb None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:52.163610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a1fd63d1-9615-44a3-a1bb-0abaa7370afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:52.172146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1171/2351] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:52.176501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b9133-e999-4e8f-b345-1a16c8bb2e92 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.186688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b9133-e999-4e8f-b345-1a16c8bb2e92 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.191304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b568967-31ba-4c37-a494-6e5e0e541601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.191987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b568967-31ba-4c37-a494-6e5e0e541601 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.192270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b568967-31ba-4c37-a494-6e5e0e541601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.192545 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b568967-31ba-4c37-a494-6e5e0e541601 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.193292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b568967-31ba-4c37-a494-6e5e0e541601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:48:52.193598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b568967-31ba-4c37-a494-6e5e0e541601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.195226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b568967-31ba-4c37-a494-6e5e0e541601 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.206712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1181/2352] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:48:52.210897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd63d1-9615-44a3-a1bb-0abaa7370afb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.220417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd63d1-9615-44a3-a1bb-0abaa7370afb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.225903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af6776-f423-4fef-b93d-595384497ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.226560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6776-f423-4fef-b93d-595384497ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.226845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6776-f423-4fef-b93d-595384497ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.227122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6776-f423-4fef-b93d-595384497ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.227935 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af6776-f423-4fef-b93d-595384497ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.229850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af6776-f423-4fef-b93d-595384497ae9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.256704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1172/2353] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:48:52 2022] POST /identity/v3/projects => generated 407 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:52.260797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b568967-31ba-4c37-a494-6e5e0e541601 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.270878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b568967-31ba-4c37-a494-6e5e0e541601 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.331998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410afc9-ce3e-4b17-9570-61756021d8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.332679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410afc9-ce3e-4b17-9570-61756021d8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.332957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410afc9-ce3e-4b17-9570-61756021d8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.333240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410afc9-ce3e-4b17-9570-61756021d8f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.333931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410afc9-ce3e-4b17-9570-61756021d8f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.335952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410afc9-ce3e-4b17-9570-61756021d8f1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.356872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1182/2354] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:48:52 2022] POST /identity/v3/users => generated 304 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:52.361012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af6776-f423-4fef-b93d-595384497ae9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.371491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af6776-f423-4fef-b93d-595384497ae9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.376445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb381c-70a8-45f1-8afd-289a5144c49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.377105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb381c-70a8-45f1-8afd-289a5144c49a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.377395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb381c-70a8-45f1-8afd-289a5144c49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.377672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb381c-70a8-45f1-8afd-289a5144c49a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.378402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb381c-70a8-45f1-8afd-289a5144c49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.380055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb381c-70a8-45f1-8afd-289a5144c49a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.391632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1173/2355] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:48:52.395554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410afc9-ce3e-4b17-9570-61756021d8f1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.405997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410afc9-ce3e-4b17-9570-61756021d8f1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.411477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebebbda-09db-4bd5-b233-2a4fd515e953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.412155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebebbda-09db-4bd5-b233-2a4fd515e953 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.412436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebebbda-09db-4bd5-b233-2a4fd515e953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.412711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebebbda-09db-4bd5-b233-2a4fd515e953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.413433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebebbda-09db-4bd5-b233-2a4fd515e953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.415042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebebbda-09db-4bd5-b233-2a4fd515e953 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.425495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1183/2356] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:48:52.429566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb381c-70a8-45f1-8afd-289a5144c49a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.440735 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb381c-70a8-45f1-8afd-289a5144c49a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.446770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a525e-f92b-4210-9e2e-a2b9286aef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.447685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a525e-f92b-4210-9e2e-a2b9286aef22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.447971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a525e-f92b-4210-9e2e-a2b9286aef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.448249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a525e-f92b-4210-9e2e-a2b9286aef22 admin admin] PATH_INFO: `/v3/projects/9ad3bf93d28542489852a04aa9ed8be4/users/835145a1a9754b4999ce729add921675/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.464015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a525e-f92b-4210-9e2e-a2b9286aef22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad3bf93d28542489852a04aa9ed8be4, user_id=835145a1a9754b4999ce729add921675, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.466717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a525e-f92b-4210-9e2e-a2b9286aef22 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.482654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1174/2357] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:48:52 2022] PUT /identity/v3/projects/9ad3bf93d28542489852a04aa9ed8be4/users/835145a1a9754b4999ce729add921675/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:48:52.493009 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6d003-3538-46dc-9295-245a31caa7a6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.493297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6d003-3538-46dc-9295-245a31caa7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.493598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6d003-3538-46dc-9295-245a31caa7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.521318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bfb6d003-3538-46dc-9295-245a31caa7a6 None None] MFA Rules not processed for user `835145a1a9754b4999ce729add921675`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:52.522544 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bfb6d003-3538-46dc-9295-245a31caa7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:52.541681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:52.591055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1184/2358] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:52.595643 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a525e-f92b-4210-9e2e-a2b9286aef22 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.601598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:52.661021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a525e-f92b-4210-9e2e-a2b9286aef22 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.665955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032725f9-6bcf-4953-a4e6-6cbec62e5717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.666624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032725f9-6bcf-4953-a4e6-6cbec62e5717 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.666911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032725f9-6bcf-4953-a4e6-6cbec62e5717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.667189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032725f9-6bcf-4953-a4e6-6cbec62e5717 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.667916 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032725f9-6bcf-4953-a4e6-6cbec62e5717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.669841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032725f9-6bcf-4953-a4e6-6cbec62e5717 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.697975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1175/2359] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:48:52 2022] POST /identity/v3/projects => generated 409 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:48:52.702046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6d003-3538-46dc-9295-245a31caa7a6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.713103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6d003-3538-46dc-9295-245a31caa7a6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.767967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3532a5-9d91-4f63-9212-6cb6149248e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.768671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3532a5-9d91-4f63-9212-6cb6149248e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.768958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3532a5-9d91-4f63-9212-6cb6149248e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.769236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3532a5-9d91-4f63-9212-6cb6149248e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.769940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3532a5-9d91-4f63-9212-6cb6149248e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.771915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3532a5-9d91-4f63-9212-6cb6149248e5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.793372 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1185/2360] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:48:52 2022] POST /identity/v3/users => generated 311 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:48:52.797437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032725f9-6bcf-4953-a4e6-6cbec62e5717 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.807915 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032725f9-6bcf-4953-a4e6-6cbec62e5717 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.813061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e801130-ebfc-42dc-9643-27c2b90e99dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.813723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e801130-ebfc-42dc-9643-27c2b90e99dd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.814006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e801130-ebfc-42dc-9643-27c2b90e99dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.814280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e801130-ebfc-42dc-9643-27c2b90e99dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.814999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e801130-ebfc-42dc-9643-27c2b90e99dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.816637 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e801130-ebfc-42dc-9643-27c2b90e99dd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.827395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1176/2361] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:48:52.831986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3532a5-9d91-4f63-9212-6cb6149248e5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.846677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3532a5-9d91-4f63-9212-6cb6149248e5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.853465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddac4a3-7754-4174-bb39-abdbc48cba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.854405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddac4a3-7754-4174-bb39-abdbc48cba51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.854696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddac4a3-7754-4174-bb39-abdbc48cba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.854973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddac4a3-7754-4174-bb39-abdbc48cba51 admin admin] PATH_INFO: `/v3/projects/fd458c6bae314bd4824476b66e76ec88/users/94c573f247f24952b1a02a9d948cfb3e/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.870501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddac4a3-7754-4174-bb39-abdbc48cba51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd458c6bae314bd4824476b66e76ec88, user_id=94c573f247f24952b1a02a9d948cfb3e, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.873294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddac4a3-7754-4174-bb39-abdbc48cba51 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.888893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1186/2362] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:48:52 2022] PUT /identity/v3/projects/fd458c6bae314bd4824476b66e76ec88/users/94c573f247f24952b1a02a9d948cfb3e/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:48:52.893040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e801130-ebfc-42dc-9643-27c2b90e99dd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:52.903627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e801130-ebfc-42dc-9643-27c2b90e99dd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:52.971667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdb5e6-6ec6-456e-81b9-e161493485f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:52.972514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdb5e6-6ec6-456e-81b9-e161493485f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:52.972804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdb5e6-6ec6-456e-81b9-e161493485f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:52.973093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdb5e6-6ec6-456e-81b9-e161493485f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:52.973837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdb5e6-6ec6-456e-81b9-e161493485f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:52.975470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdb5e6-6ec6-456e-81b9-e161493485f9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:52.986421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1177/2363] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:48:52.990450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddac4a3-7754-4174-bb39-abdbc48cba51 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:53.002908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddac4a3-7754-4174-bb39-abdbc48cba51 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:53.009966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b92b1-db9a-4f36-9bc7-3e0dd7738c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:53.010938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b92b1-db9a-4f36-9bc7-3e0dd7738c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:53.011297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b92b1-db9a-4f36-9bc7-3e0dd7738c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:53.011680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b92b1-db9a-4f36-9bc7-3e0dd7738c65 admin admin] PATH_INFO: `/v3/projects/fd458c6bae314bd4824476b66e76ec88/users/94c573f247f24952b1a02a9d948cfb3e/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:53.013766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b92b1-db9a-4f36-9bc7-3e0dd7738c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd458c6bae314bd4824476b66e76ec88, user_id=94c573f247f24952b1a02a9d948cfb3e, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:53.016513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b92b1-db9a-4f36-9bc7-3e0dd7738c65 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:53.027274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1187/2364] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:48:52 2022] PUT /identity/v3/projects/fd458c6bae314bd4824476b66e76ec88/users/94c573f247f24952b1a02a9d948cfb3e/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:48:53.038750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10caf38-9fb9-44e2-8013-e38e9734c395 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:53.039045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10caf38-9fb9-44e2-8013-e38e9734c395 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:53.039330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10caf38-9fb9-44e2-8013-e38e9734c395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:53.067142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a10caf38-9fb9-44e2-8013-e38e9734c395 None None] MFA Rules not processed for user `94c573f247f24952b1a02a9d948cfb3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:48:53.068405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a10caf38-9fb9-44e2-8013-e38e9734c395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:53.133162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:48:53.137154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1178/2365] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:48:53.148405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b92b1-db9a-4f36-9bc7-3e0dd7738c65 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:53.150557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-673b92b1-db9a-4f36-9bc7-3e0dd7738c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:48:53.213306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b92b1-db9a-4f36-9bc7-3e0dd7738c65 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:53.217427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa858e1e-99bb-45ef-af7b-f7b478dae0e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:53.218092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa858e1e-99bb-45ef-af7b-f7b478dae0e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:53.218378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa858e1e-99bb-45ef-af7b-f7b478dae0e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:53.218659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa858e1e-99bb-45ef-af7b-f7b478dae0e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:53.228960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa858e1e-99bb-45ef-af7b-f7b478dae0e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:53.231215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa858e1e-99bb-45ef-af7b-f7b478dae0e8 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:53.245699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1188/2366] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:53 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:48:53.412701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10caf38-9fb9-44e2-8013-e38e9734c395 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:48:53.423838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10caf38-9fb9-44e2-8013-e38e9734c395 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:48:53.428554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a2df3-d405-4817-abcf-50a5fb7438b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:48:53.429227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a2df3-d405-4817-abcf-50a5fb7438b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:48:53.429511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a2df3-d405-4817-abcf-50a5fb7438b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:48:53.429808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a2df3-d405-4817-abcf-50a5fb7438b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:48:53.495150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a2df3-d405-4817-abcf-50a5fb7438b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:48:53.496559 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a2df3-d405-4817-abcf-50a5fb7438b3 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:48:53.505588 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1179/2367] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:48:53 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:49:02.960318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa858e1e-99bb-45ef-af7b-f7b478dae0e8 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:02.980643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:03.030523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa858e1e-99bb-45ef-af7b-f7b478dae0e8 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:03.035146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:03.036068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:03.036379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:03.036657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] PATH_INFO: `/v3/users/835145a1a9754b4999ce729add921675` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:03.037930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=835145a1a9754b4999ce729add921675)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:03.039967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:03.091294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835145a1a9754b4999ce729add921675'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.091985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835145a1a9754b4999ce729add921675'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.104566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835145a1a9754b4999ce729add921675'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.111913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835145a1a9754b4999ce729add921675'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.120446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835145a1a9754b4999ce729add921675'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.127316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1189/2368] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:49:02 2022] DELETE /identity/v3/users/835145a1a9754b4999ce729add921675 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:03.131955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a342f-77c9-4ec9-8d85-aa42f01a47e6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:03.132416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a342f-77c9-4ec9-8d85-aa42f01a47e6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:03.132565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a342f-77c9-4ec9-8d85-aa42f01a47e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:03.150020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f44a342f-77c9-4ec9-8d85-aa42f01a47e6 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:03.151231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f44a342f-77c9-4ec9-8d85-aa42f01a47e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:03.214989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1180/2369] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:03.224824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:03.226280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:03.285205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ef197-2266-47aa-a203-94f4c09a9047 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:03.288752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b4845-8ca4-47b3-b3d0-132be759b5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:03.289413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b4845-8ca4-47b3-b3d0-132be759b5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:03.289705 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b4845-8ca4-47b3-b3d0-132be759b5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:03.289987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b4845-8ca4-47b3-b3d0-132be759b5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:03.298987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b4845-8ca4-47b3-b3d0-132be759b5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:03.300498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b4845-8ca4-47b3-b3d0-132be759b5e8 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:03.310427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1190/2370] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:49:03.598311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a342f-77c9-4ec9-8d85-aa42f01a47e6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:03.600059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f44a342f-77c9-4ec9-8d85-aa42f01a47e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:03.616771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a342f-77c9-4ec9-8d85-aa42f01a47e6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:03.625076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:03.626759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:03.627116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:03.627405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] PATH_INFO: `/v3/projects/9ad3bf93d28542489852a04aa9ed8be4` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:03.628816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ad3bf93d28542489852a04aa9ed8be4)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:03.630838 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:03.681905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ad3bf93d28542489852a04aa9ed8be4', 'request_id': 'req-cd5b2203-f837-461b-9d28-3773e97a3604'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.688959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ad3bf93d28542489852a04aa9ed8be4', 'request_id': 'req-cd5b2203-f837-461b-9d28-3773e97a3604'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.695351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ad3bf93d28542489852a04aa9ed8be4', 'request_id': 'req-cd5b2203-f837-461b-9d28-3773e97a3604'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.709354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] The token cache is being invalidate because project 9ad3bf93d28542489852a04aa9ed8be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:49:03.709649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.710371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.712165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1181/2371] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:49:03 2022] DELETE /identity/v3/projects/9ad3bf93d28542489852a04aa9ed8be4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:49:03.716326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b4845-8ca4-47b3-b3d0-132be759b5e8 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:03.726258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b4845-8ca4-47b3-b3d0-132be759b5e8 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:03.762000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:03.787451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:03.788356 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:03.788647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:03.788931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] PATH_INFO: `/v3/users/94c573f247f24952b1a02a9d948cfb3e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:03.790143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c573f247f24952b1a02a9d948cfb3e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:03.792110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:03.840791 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c573f247f24952b1a02a9d948cfb3e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.841374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c573f247f24952b1a02a9d948cfb3e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.853264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c573f247f24952b1a02a9d948cfb3e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.860222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c573f247f24952b1a02a9d948cfb3e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.869067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1603c25d-7828-4fd6-a764-e76c0998bd41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c573f247f24952b1a02a9d948cfb3e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:03.875556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1191/2372] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:49:03 2022] DELETE /identity/v3/users/94c573f247f24952b1a02a9d948cfb3e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:04.165490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:04.167264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:04.181003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b2203-f837-461b-9d28-3773e97a3604 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:04.205747 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:04.248548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:04.249433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.249720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.250027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] PATH_INFO: `/v3/projects/fd458c6bae314bd4824476b66e76ec88` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.251317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd458c6bae314bd4824476b66e76ec88)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:04.253282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:04.302761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd458c6bae314bd4824476b66e76ec88', 'request_id': 'req-4cde44bd-db95-449f-afdb-b3e65c50e69f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:04.308844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd458c6bae314bd4824476b66e76ec88', 'request_id': 'req-4cde44bd-db95-449f-afdb-b3e65c50e69f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:04.315973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd458c6bae314bd4824476b66e76ec88', 'request_id': 'req-4cde44bd-db95-449f-afdb-b3e65c50e69f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:04.333525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] The token cache is being invalidate because project fd458c6bae314bd4824476b66e76ec88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:49:04.333850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:04.334837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4cde44bd-db95-449f-afdb-b3e65c50e69f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:04.337322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1182/2373] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:49:04 2022] DELETE /identity/v3/projects/fd458c6bae314bd4824476b66e76ec88 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:04.342571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b266a-54e9-4d3f-b7f0-2c448d2a1b21 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.343058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b266a-54e9-4d3f-b7f0-2c448d2a1b21 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.343155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b266a-54e9-4d3f-b7f0-2c448d2a1b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.363436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9d9b266a-54e9-4d3f-b7f0-2c448d2a1b21 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:04.364614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9d9b266a-54e9-4d3f-b7f0-2c448d2a1b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:04.432050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1192/2374] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:04.440512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61acc2-168e-46a4-a073-bfd92b8280ff None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.440806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61acc2-168e-46a4-a073-bfd92b8280ff None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.441088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61acc2-168e-46a4-a073-bfd92b8280ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.461130 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7f61acc2-168e-46a4-a073-bfd92b8280ff None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:04.462358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7f61acc2-168e-46a4-a073-bfd92b8280ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:04.471868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1183/2375] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:04.476254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b266a-54e9-4d3f-b7f0-2c448d2a1b21 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:04.487986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b266a-54e9-4d3f-b7f0-2c448d2a1b21 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:04.496359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3332cac-caa0-4527-98eb-8c12074c454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:04.497025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3332cac-caa0-4527-98eb-8c12074c454f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.497308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3332cac-caa0-4527-98eb-8c12074c454f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.497582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3332cac-caa0-4527-98eb-8c12074c454f admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.498313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3332cac-caa0-4527-98eb-8c12074c454f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:49:04.498619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3332cac-caa0-4527-98eb-8c12074c454f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:04.500306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3332cac-caa0-4527-98eb-8c12074c454f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:04.514113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1193/2376] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:49:04.518658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61acc2-168e-46a4-a073-bfd92b8280ff None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:04.528782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61acc2-168e-46a4-a073-bfd92b8280ff None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:04.534836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba29041-2e1a-4ac0-b475-7160138e4ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:04.535492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba29041-2e1a-4ac0-b475-7160138e4ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.535834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba29041-2e1a-4ac0-b475-7160138e4ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.536113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba29041-2e1a-4ac0-b475-7160138e4ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.536800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba29041-2e1a-4ac0-b475-7160138e4ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:04.538786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba29041-2e1a-4ac0-b475-7160138e4ac9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:04.566487 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1184/2377] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:49:04 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:49:04.570733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3332cac-caa0-4527-98eb-8c12074c454f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:04.581267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3332cac-caa0-4527-98eb-8c12074c454f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:04.625272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:04.643619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ddeb0-2a75-410b-bd56-c3b991834b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:04.644305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ddeb0-2a75-410b-bd56-c3b991834b52 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.644596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ddeb0-2a75-410b-bd56-c3b991834b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.644878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ddeb0-2a75-410b-bd56-c3b991834b52 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.645584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ddeb0-2a75-410b-bd56-c3b991834b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:04.647593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ddeb0-2a75-410b-bd56-c3b991834b52 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:04.669746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1194/2378] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:49:04 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:49:04.673958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba29041-2e1a-4ac0-b475-7160138e4ac9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:04.684610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba29041-2e1a-4ac0-b475-7160138e4ac9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:04.690018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb939692-5591-4894-a41e-db38fb341038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:04.690687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb939692-5591-4894-a41e-db38fb341038 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.690976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb939692-5591-4894-a41e-db38fb341038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.691260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb939692-5591-4894-a41e-db38fb341038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.692022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb939692-5591-4894-a41e-db38fb341038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:04.693635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb939692-5591-4894-a41e-db38fb341038 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:04.704462 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1185/2379] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:49:04.708552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ddeb0-2a75-410b-bd56-c3b991834b52 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:04.719020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ddeb0-2a75-410b-bd56-c3b991834b52 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:04.724561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715be136-3ff2-4718-a612-6c72d2909c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:04.725240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715be136-3ff2-4718-a612-6c72d2909c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.725524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715be136-3ff2-4718-a612-6c72d2909c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.725804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715be136-3ff2-4718-a612-6c72d2909c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.726538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715be136-3ff2-4718-a612-6c72d2909c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:04.728190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715be136-3ff2-4718-a612-6c72d2909c86 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:04.738851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1195/2380] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:49:04.742955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb939692-5591-4894-a41e-db38fb341038 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:04.753406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb939692-5591-4894-a41e-db38fb341038 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:04.759160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e517b-34f3-4b0c-9c33-342c901850f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:04.760157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e517b-34f3-4b0c-9c33-342c901850f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.760446 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e517b-34f3-4b0c-9c33-342c901850f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.760772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e517b-34f3-4b0c-9c33-342c901850f7 admin admin] PATH_INFO: `/v3/projects/f37b9213635a44769022c20845ba5478/users/407d6590606a4e7889534d93d2386eb2/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.776270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e517b-34f3-4b0c-9c33-342c901850f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37b9213635a44769022c20845ba5478, user_id=407d6590606a4e7889534d93d2386eb2, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:04.778947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e517b-34f3-4b0c-9c33-342c901850f7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:04.793183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1186/2381] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:49:04 2022] PUT /identity/v3/projects/f37b9213635a44769022c20845ba5478/users/407d6590606a4e7889534d93d2386eb2/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:04.803929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f1464-5879-435b-8460-2affe47f1c58 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:04.804225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f1464-5879-435b-8460-2affe47f1c58 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:04.804538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f1464-5879-435b-8460-2affe47f1c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:04.831395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ba9f1464-5879-435b-8460-2affe47f1c58 None None] MFA Rules not processed for user `407d6590606a4e7889534d93d2386eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:04.832639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ba9f1464-5879-435b-8460-2affe47f1c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:04.904313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1196/2382] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:49:04.916891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e517b-34f3-4b0c-9c33-342c901850f7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:04.918617 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f23e517b-34f3-4b0c-9c33-342c901850f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:04.989094 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:04.995887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e517b-34f3-4b0c-9c33-342c901850f7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:05.005840 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd5e04-7729-47f0-8239-908abc1b556d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:05.006628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd5e04-7729-47f0-8239-908abc1b556d service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:05.006813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd5e04-7729-47f0-8239-908abc1b556d service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:05.007099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd5e04-7729-47f0-8239-908abc1b556d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:05.016417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd5e04-7729-47f0-8239-908abc1b556d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:05.017953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd5e04-7729-47f0-8239-908abc1b556d service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:05.026955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1187/2383] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:49:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:49:09.105866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f1464-5879-435b-8460-2affe47f1c58 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:09.173343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f1464-5879-435b-8460-2affe47f1c58 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:09.178421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:09.179304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:09.179624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:09.179905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] PATH_INFO: `/v3/users/407d6590606a4e7889534d93d2386eb2` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:09.181176 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=407d6590606a4e7889534d93d2386eb2)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:09.183151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:09.231518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '407d6590606a4e7889534d93d2386eb2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.232185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '407d6590606a4e7889534d93d2386eb2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.237347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:09.245873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '407d6590606a4e7889534d93d2386eb2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.253463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '407d6590606a4e7889534d93d2386eb2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.262898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '407d6590606a4e7889534d93d2386eb2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.269466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1197/2384] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:49:09 2022] DELETE /identity/v3/users/407d6590606a4e7889534d93d2386eb2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:09.274105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56de1d-ab71-4d62-b67e-343bdf3b6877 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:09.274593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56de1d-ab71-4d62-b67e-343bdf3b6877 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:09.274685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56de1d-ab71-4d62-b67e-343bdf3b6877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:09.292302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fb56de1d-ab71-4d62-b67e-343bdf3b6877 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:09.293475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fb56de1d-ab71-4d62-b67e-343bdf3b6877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:09.356473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1188/2385] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:49:09.366270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:09.367921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:09.426750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8eea9-e9a4-454d-a25a-4db89dbf4441 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:09.432454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617cee9-f5e8-4f04-a2af-0dd3e0bdab53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:09.433597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617cee9-f5e8-4f04-a2af-0dd3e0bdab53 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:09.434100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617cee9-f5e8-4f04-a2af-0dd3e0bdab53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:09.434608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617cee9-f5e8-4f04-a2af-0dd3e0bdab53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:09.445945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617cee9-f5e8-4f04-a2af-0dd3e0bdab53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:09.447487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617cee9-f5e8-4f04-a2af-0dd3e0bdab53 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:09.463208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1198/2386] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:49:09.760854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56de1d-ab71-4d62-b67e-343bdf3b6877 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:09.762528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fb56de1d-ab71-4d62-b67e-343bdf3b6877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:09.775881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56de1d-ab71-4d62-b67e-343bdf3b6877 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:09.781658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:09.782579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:09.782862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:09.783135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] PATH_INFO: `/v3/projects/f37b9213635a44769022c20845ba5478` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:09.784423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f37b9213635a44769022c20845ba5478)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:09.786514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:09.842434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f37b9213635a44769022c20845ba5478', 'request_id': 'req-b523e1c6-0079-4518-9f91-b3f56cb3d228'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.852237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f37b9213635a44769022c20845ba5478', 'request_id': 'req-b523e1c6-0079-4518-9f91-b3f56cb3d228'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.861248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f37b9213635a44769022c20845ba5478', 'request_id': 'req-b523e1c6-0079-4518-9f91-b3f56cb3d228'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.879066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] The token cache is being invalidate because project f37b9213635a44769022c20845ba5478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:49:09.879383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.880293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b523e1c6-0079-4518-9f91-b3f56cb3d228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:09.882253 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1189/2387] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:49:09 2022] DELETE /identity/v3/projects/f37b9213635a44769022c20845ba5478 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:09.887440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eb894-8ee3-442a-95be-9ab9ea29e6cd None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:09.887805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eb894-8ee3-442a-95be-9ab9ea29e6cd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:09.888166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eb894-8ee3-442a-95be-9ab9ea29e6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:09.908887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cf8eb894-8ee3-442a-95be-9ab9ea29e6cd None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:09.910407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cf8eb894-8ee3-442a-95be-9ab9ea29e6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:09.975901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1199/2388] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:09.989474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf925d1e-c838-49de-8017-d0f59a3efc4f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:09.989784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf925d1e-c838-49de-8017-d0f59a3efc4f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:09.990054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf925d1e-c838-49de-8017-d0f59a3efc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.011405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cf925d1e-c838-49de-8017-d0f59a3efc4f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:10.012664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cf925d1e-c838-49de-8017-d0f59a3efc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:10.021231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1190/2389] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:10.025661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8eb894-8ee3-442a-95be-9ab9ea29e6cd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:10.036692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8eb894-8ee3-442a-95be-9ab9ea29e6cd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:10.042107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f3fc-4d12-4889-b98b-47b78c13f369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:10.042795 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f3fc-4d12-4889-b98b-47b78c13f369 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.043086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f3fc-4d12-4889-b98b-47b78c13f369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.043392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f3fc-4d12-4889-b98b-47b78c13f369 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.044165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f3fc-4d12-4889-b98b-47b78c13f369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:49:10.044477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f3fc-4d12-4889-b98b-47b78c13f369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:10.046118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f3fc-4d12-4889-b98b-47b78c13f369 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:10.062519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1200/2390] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:49:10.067024 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf925d1e-c838-49de-8017-d0f59a3efc4f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:10.078363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf925d1e-c838-49de-8017-d0f59a3efc4f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:10.083284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc1c1b-8f56-4f7d-b90f-79b9d1ddc5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:10.083967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc1c1b-8f56-4f7d-b90f-79b9d1ddc5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.084249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc1c1b-8f56-4f7d-b90f-79b9d1ddc5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.084522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc1c1b-8f56-4f7d-b90f-79b9d1ddc5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.085243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc1c1b-8f56-4f7d-b90f-79b9d1ddc5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:10.087138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc1c1b-8f56-4f7d-b90f-79b9d1ddc5fb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:10.123405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1191/2391] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:49:10 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:49:10.129426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f3fc-4d12-4889-b98b-47b78c13f369 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:10.146623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f3fc-4d12-4889-b98b-47b78c13f369 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:10.212128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16ec60-16e6-41fe-9d26-11d59bc7a9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:10.212931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16ec60-16e6-41fe-9d26-11d59bc7a9fd admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.213220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16ec60-16e6-41fe-9d26-11d59bc7a9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.213613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16ec60-16e6-41fe-9d26-11d59bc7a9fd admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.214847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16ec60-16e6-41fe-9d26-11d59bc7a9fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:10.217268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16ec60-16e6-41fe-9d26-11d59bc7a9fd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:10.225276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:10.242858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1201/2392] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:49:10 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:49:10.247255 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc1c1b-8f56-4f7d-b90f-79b9d1ddc5fb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:10.259112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc1c1b-8f56-4f7d-b90f-79b9d1ddc5fb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:10.265336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ec878-3f78-4b08-b35a-e83881e2587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:10.266004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ec878-3f78-4b08-b35a-e83881e2587e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.266299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ec878-3f78-4b08-b35a-e83881e2587e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.266581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ec878-3f78-4b08-b35a-e83881e2587e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.267309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ec878-3f78-4b08-b35a-e83881e2587e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:10.268956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ec878-3f78-4b08-b35a-e83881e2587e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:10.282028 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1192/2393] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:49:10.286225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16ec60-16e6-41fe-9d26-11d59bc7a9fd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:10.296702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16ec60-16e6-41fe-9d26-11d59bc7a9fd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:10.302158 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0083c-38c0-49ad-9b45-461e6cb4a033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:10.302870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0083c-38c0-49ad-9b45-461e6cb4a033 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.303208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0083c-38c0-49ad-9b45-461e6cb4a033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.303529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0083c-38c0-49ad-9b45-461e6cb4a033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.304256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0083c-38c0-49ad-9b45-461e6cb4a033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:10.305891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0083c-38c0-49ad-9b45-461e6cb4a033 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:10.317250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1202/2394] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:49:10.321485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ec878-3f78-4b08-b35a-e83881e2587e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:10.331805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ec878-3f78-4b08-b35a-e83881e2587e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:10.337285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ed8a7-5139-41be-a249-7c66e0881bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:10.338179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ed8a7-5139-41be-a249-7c66e0881bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.338461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ed8a7-5139-41be-a249-7c66e0881bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.338742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ed8a7-5139-41be-a249-7c66e0881bd5 admin admin] PATH_INFO: `/v3/projects/61f4e314c93b4f208f6ca412be03bda4/users/b44fdfa8473e45a498bf65e3b3c78ab2/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.353973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ed8a7-5139-41be-a249-7c66e0881bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f4e314c93b4f208f6ca412be03bda4, user_id=b44fdfa8473e45a498bf65e3b3c78ab2, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:10.356723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ed8a7-5139-41be-a249-7c66e0881bd5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:10.374044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1193/2395] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:49:10 2022] PUT /identity/v3/projects/61f4e314c93b4f208f6ca412be03bda4/users/b44fdfa8473e45a498bf65e3b3c78ab2/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:10.389837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a985725-7a06-42de-83ca-b26dc617712d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.390134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a985725-7a06-42de-83ca-b26dc617712d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.390421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a985725-7a06-42de-83ca-b26dc617712d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.419593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1a985725-7a06-42de-83ca-b26dc617712d None None] MFA Rules not processed for user `b44fdfa8473e45a498bf65e3b3c78ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:10.420910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1a985725-7a06-42de-83ca-b26dc617712d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:10.504830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1203/2396] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:10.516403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ed8a7-5139-41be-a249-7c66e0881bd5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:10.518722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fc9ed8a7-5139-41be-a249-7c66e0881bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:10.534827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:10.583296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ed8a7-5139-41be-a249-7c66e0881bd5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:10.587496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceaa10-aff3-474e-8ca9-c4971a171774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:10.588189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceaa10-aff3-474e-8ca9-c4971a171774 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.588476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceaa10-aff3-474e-8ca9-c4971a171774 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.588752 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceaa10-aff3-474e-8ca9-c4971a171774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.597442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceaa10-aff3-474e-8ca9-c4971a171774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:10.598911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceaa10-aff3-474e-8ca9-c4971a171774 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:10.607962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1194/2397] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:49:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:49:10.632167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a985725-7a06-42de-83ca-b26dc617712d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:10.695754 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a985725-7a06-42de-83ca-b26dc617712d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:10.700442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:10.701318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.701605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.701882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] PATH_INFO: `/v3/users/b44fdfa8473e45a498bf65e3b3c78ab2` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.703074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b44fdfa8473e45a498bf65e3b3c78ab2)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:10.705081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:10.761245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b44fdfa8473e45a498bf65e3b3c78ab2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:10.761918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b44fdfa8473e45a498bf65e3b3c78ab2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:10.773829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b44fdfa8473e45a498bf65e3b3c78ab2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:10.779681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:10.781782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b44fdfa8473e45a498bf65e3b3c78ab2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:10.790887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b44fdfa8473e45a498bf65e3b3c78ab2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:10.797512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1204/2398] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:49:10 2022] DELETE /identity/v3/users/b44fdfa8473e45a498bf65e3b3c78ab2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:10.802159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1426b-307c-4c61-b91c-89757fb947d6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.802454 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1426b-307c-4c61-b91c-89757fb947d6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.802761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1426b-307c-4c61-b91c-89757fb947d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.822625 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6ea1426b-307c-4c61-b91c-89757fb947d6 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:10.823907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6ea1426b-307c-4c61-b91c-89757fb947d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:10.888880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1195/2399] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:49:10.898697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:10.900214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:10.960660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e5787-a8d0-438f-b4ab-a4f01c65de9f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:10.965827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbaf02-b06c-4b2d-b5d6-f2948cd16c42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:10.966494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbaf02-b06c-4b2d-b5d6-f2948cd16c42 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:10.966780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbaf02-b06c-4b2d-b5d6-f2948cd16c42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:10.967053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbaf02-b06c-4b2d-b5d6-f2948cd16c42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:10.977672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbaf02-b06c-4b2d-b5d6-f2948cd16c42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:10.979196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbaf02-b06c-4b2d-b5d6-f2948cd16c42 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:10.989231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1205/2400] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:49:11.265378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1426b-307c-4c61-b91c-89757fb947d6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:11.266862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6ea1426b-307c-4c61-b91c-89757fb947d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:11.279706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1426b-307c-4c61-b91c-89757fb947d6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:11.285315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:11.286203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.286489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.286768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] PATH_INFO: `/v3/projects/61f4e314c93b4f208f6ca412be03bda4` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.288112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=61f4e314c93b4f208f6ca412be03bda4)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:11.290112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:11.340373 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61f4e314c93b4f208f6ca412be03bda4', 'request_id': 'req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:11.346491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61f4e314c93b4f208f6ca412be03bda4', 'request_id': 'req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:11.353420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61f4e314c93b4f208f6ca412be03bda4', 'request_id': 'req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:11.366038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] The token cache is being invalidate because project 61f4e314c93b4f208f6ca412be03bda4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:49:11.366397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:11.367184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e51b6b3-05aa-41ce-99fc-fe1904d8fc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:11.368920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1196/2401] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:49:11 2022] DELETE /identity/v3/projects/61f4e314c93b4f208f6ca412be03bda4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:11.373886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3964397-a129-4d27-b3b0-1a87eb2e91df None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.374385 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3964397-a129-4d27-b3b0-1a87eb2e91df None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.374485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3964397-a129-4d27-b3b0-1a87eb2e91df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.392362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c3964397-a129-4d27-b3b0-1a87eb2e91df None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:11.393504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c3964397-a129-4d27-b3b0-1a87eb2e91df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:11.454670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1206/2402] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:11.464278 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7a6e1-f5e7-41f3-bdc3-36eb41ec78df None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.464715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7a6e1-f5e7-41f3-bdc3-36eb41ec78df None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.465011 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7a6e1-f5e7-41f3-bdc3-36eb41ec78df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.488618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-46e7a6e1-f5e7-41f3-bdc3-36eb41ec78df None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:11.489991 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-46e7a6e1-f5e7-41f3-bdc3-36eb41ec78df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:11.499357 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1197/2403] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:11.503820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3964397-a129-4d27-b3b0-1a87eb2e91df None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:11.514505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3964397-a129-4d27-b3b0-1a87eb2e91df None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:11.521307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb057ab-5051-4758-b2e9-859f2369da18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:11.522314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb057ab-5051-4758-b2e9-859f2369da18 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.522750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb057ab-5051-4758-b2e9-859f2369da18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.523225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb057ab-5051-4758-b2e9-859f2369da18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.524526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb057ab-5051-4758-b2e9-859f2369da18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:49:11.525008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb057ab-5051-4758-b2e9-859f2369da18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:11.527949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb057ab-5051-4758-b2e9-859f2369da18 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:11.542768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1207/2404] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:49:11.548126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7a6e1-f5e7-41f3-bdc3-36eb41ec78df None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:11.560606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7a6e1-f5e7-41f3-bdc3-36eb41ec78df None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:11.565623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805a530-c77e-40af-bada-8e8d5c944d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:11.566334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805a530-c77e-40af-bada-8e8d5c944d79 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.566621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805a530-c77e-40af-bada-8e8d5c944d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.566925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805a530-c77e-40af-bada-8e8d5c944d79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.567700 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805a530-c77e-40af-bada-8e8d5c944d79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:11.569615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805a530-c77e-40af-bada-8e8d5c944d79 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:11.596208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1198/2405] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:49:11 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:49:11.600900 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb057ab-5051-4758-b2e9-859f2369da18 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:11.611824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb057ab-5051-4758-b2e9-859f2369da18 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:11.675097 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e9867-f354-4eb8-8aa5-4c7766ade2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:11.675723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e9867-f354-4eb8-8aa5-4c7766ade2fc admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.676016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e9867-f354-4eb8-8aa5-4c7766ade2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.676299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e9867-f354-4eb8-8aa5-4c7766ade2fc admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.676992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e9867-f354-4eb8-8aa5-4c7766ade2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:11.678950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e9867-f354-4eb8-8aa5-4c7766ade2fc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:11.699788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1208/2406] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:49:11 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:49:11.703808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805a530-c77e-40af-bada-8e8d5c944d79 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:11.714065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805a530-c77e-40af-bada-8e8d5c944d79 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:11.719158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218e9d3-a244-4762-8d05-8cf0e60dfa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:11.719856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218e9d3-a244-4762-8d05-8cf0e60dfa04 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.720141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218e9d3-a244-4762-8d05-8cf0e60dfa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.720421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218e9d3-a244-4762-8d05-8cf0e60dfa04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.721172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218e9d3-a244-4762-8d05-8cf0e60dfa04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:11.722763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218e9d3-a244-4762-8d05-8cf0e60dfa04 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:11.724927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:11.735022 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1199/2407] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:49:11.739111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e9867-f354-4eb8-8aa5-4c7766ade2fc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:11.749373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e9867-f354-4eb8-8aa5-4c7766ade2fc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:11.754532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927ba98-dfd5-47ab-85c2-599154878905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:11.755196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927ba98-dfd5-47ab-85c2-599154878905 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.755507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927ba98-dfd5-47ab-85c2-599154878905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.755787 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927ba98-dfd5-47ab-85c2-599154878905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.756507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927ba98-dfd5-47ab-85c2-599154878905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:11.758116 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927ba98-dfd5-47ab-85c2-599154878905 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:11.760269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:11.770903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1209/2408] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:49:11.774818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218e9d3-a244-4762-8d05-8cf0e60dfa04 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:11.785298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218e9d3-a244-4762-8d05-8cf0e60dfa04 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:11.790014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26bea1-f2e3-4b12-812e-89aaef4cfa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:11.790897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26bea1-f2e3-4b12-812e-89aaef4cfa57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.791179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26bea1-f2e3-4b12-812e-89aaef4cfa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.791474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26bea1-f2e3-4b12-812e-89aaef4cfa57 admin admin] PATH_INFO: `/v3/projects/7de3380022b04eaa9872b6c51cc0084e/users/d68e584fa9f14ffe9fbffa6a24ed4ce1/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.805907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26bea1-f2e3-4b12-812e-89aaef4cfa57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de3380022b04eaa9872b6c51cc0084e, user_id=d68e584fa9f14ffe9fbffa6a24ed4ce1, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:11.808645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26bea1-f2e3-4b12-812e-89aaef4cfa57 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:11.823434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1200/2409] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:49:11 2022] PUT /identity/v3/projects/7de3380022b04eaa9872b6c51cc0084e/users/d68e584fa9f14ffe9fbffa6a24ed4ce1/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:11.888805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64072a-022f-4c44-bdd7-6e0453f02004 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:11.889122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64072a-022f-4c44-bdd7-6e0453f02004 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:11.889416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64072a-022f-4c44-bdd7-6e0453f02004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:11.917279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3f64072a-022f-4c44-bdd7-6e0453f02004 None None] MFA Rules not processed for user `d68e584fa9f14ffe9fbffa6a24ed4ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:11.918524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3f64072a-022f-4c44-bdd7-6e0453f02004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:11.992166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1210/2410] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:12.003258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26bea1-f2e3-4b12-812e-89aaef4cfa57 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:12.005299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ef26bea1-f2e3-4b12-812e-89aaef4cfa57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:12.066570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26bea1-f2e3-4b12-812e-89aaef4cfa57 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:12.070717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057d00c-441b-46af-9af0-e55f3332e585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:12.071404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057d00c-441b-46af-9af0-e55f3332e585 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:12.071723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057d00c-441b-46af-9af0-e55f3332e585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:12.072006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057d00c-441b-46af-9af0-e55f3332e585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:12.080917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057d00c-441b-46af-9af0-e55f3332e585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:12.082365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057d00c-441b-46af-9af0-e55f3332e585 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:12.092628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1201/2411] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:49:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:49:21.988805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=33 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:22.087534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:33.439722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df59dd-ffd1-43c6-b4a9-2959320f3062 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:33.440302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df59dd-ffd1-43c6-b4a9-2959320f3062 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:33.440302 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df59dd-ffd1-43c6-b4a9-2959320f3062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:33.468306 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a8df59dd-ffd1-43c6-b4a9-2959320f3062 None None] MFA Rules not processed for user `0b22e96941dd414da4fa22ed4bd36984`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:33.469493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a8df59dd-ffd1-43c6-b4a9-2959320f3062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:33.543032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1211/2412] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:36.686437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057d00c-441b-46af-9af0-e55f3332e585 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:36.802316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057d00c-441b-46af-9af0-e55f3332e585 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:36.807468 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:36.808395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:36.808682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:36.808973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] PATH_INFO: `/v3/users/d68e584fa9f14ffe9fbffa6a24ed4ce1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:36.810562 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d68e584fa9f14ffe9fbffa6a24ed4ce1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:36.812570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:36.860274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd68e584fa9f14ffe9fbffa6a24ed4ce1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:36.860931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd68e584fa9f14ffe9fbffa6a24ed4ce1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:36.874312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd68e584fa9f14ffe9fbffa6a24ed4ce1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:36.883178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd68e584fa9f14ffe9fbffa6a24ed4ce1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:36.884020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd68e584fa9f14ffe9fbffa6a24ed4ce1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:36.898009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1202/2413] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:49:36 2022] DELETE /identity/v3/users/d68e584fa9f14ffe9fbffa6a24ed4ce1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:36.902423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3126c4-becf-47a2-a888-c61d96ac1699 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:36.902974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3126c4-becf-47a2-a888-c61d96ac1699 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:36.902974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3126c4-becf-47a2-a888-c61d96ac1699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:36.944011 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3a3126c4-becf-47a2-a888-c61d96ac1699 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:36.945407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3a3126c4-becf-47a2-a888-c61d96ac1699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:37.003973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1212/2414] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:37.015320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:37.016908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:37.100481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:37.151326 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501af20b-13cd-4366-b621-b1adb60428dd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:37.155173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1e783-4149-43a0-8029-dd1cbd8862b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:37.156030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1e783-4149-43a0-8029-dd1cbd8862b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:37.156323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1e783-4149-43a0-8029-dd1cbd8862b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:37.156609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1e783-4149-43a0-8029-dd1cbd8862b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:37.168928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1e783-4149-43a0-8029-dd1cbd8862b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:37.170943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1e783-4149-43a0-8029-dd1cbd8862b0 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:37.183337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1203/2415] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:49:37.527843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3126c4-becf-47a2-a888-c61d96ac1699 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:37.530100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3a3126c4-becf-47a2-a888-c61d96ac1699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:37.548589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3126c4-becf-47a2-a888-c61d96ac1699 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:37.553437 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:37.554340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:37.554770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:37.555047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] PATH_INFO: `/v3/projects/7de3380022b04eaa9872b6c51cc0084e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:37.556636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7de3380022b04eaa9872b6c51cc0084e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:37.559190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:37.609510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7de3380022b04eaa9872b6c51cc0084e', 'request_id': 'req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:37.618333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7de3380022b04eaa9872b6c51cc0084e', 'request_id': 'req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:37.619852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:37.625640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7de3380022b04eaa9872b6c51cc0084e', 'request_id': 'req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:37.639090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] The token cache is being invalidate because project 7de3380022b04eaa9872b6c51cc0084e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:49:37.639385 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:37.640077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:49:37.642282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1213/2416] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:49:37 2022] DELETE /identity/v3/projects/7de3380022b04eaa9872b6c51cc0084e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:49:47.177042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=16 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:47.635524 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=2,keystone:SELECT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:49:56.617561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163575d-9065-4cc2-9e17-a3f36f0cb982 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:56.618039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163575d-9065-4cc2-9e17-a3f36f0cb982 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:56.618135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163575d-9065-4cc2-9e17-a3f36f0cb982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:56.648987 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2163575d-9065-4cc2-9e17-a3f36f0cb982 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:49:56.650209 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2163575d-9065-4cc2-9e17-a3f36f0cb982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:56.713719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1204/2417] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 13 20:49:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:49:56.761815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:49:56.763310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:49:56.836324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc99dc-92ea-4f12-bb42-5c2e12615f6d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:49:56.842198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a24127-686c-4176-8c98-5a9d7c8f439c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:49:56.842857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a24127-686c-4176-8c98-5a9d7c8f439c service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:49:56.843145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a24127-686c-4176-8c98-5a9d7c8f439c service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:49:56.843428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a24127-686c-4176-8c98-5a9d7c8f439c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:49:56.851898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a24127-686c-4176-8c98-5a9d7c8f439c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:49:56.853514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a24127-686c-4176-8c98-5a9d7c8f439c service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:49:56.861640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1214/2418] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:49:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:50:00.519584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163575d-9065-4cc2-9e17-a3f36f0cb982 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:00.521284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2163575d-9065-4cc2-9e17-a3f36f0cb982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:00.639541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163575d-9065-4cc2-9e17-a3f36f0cb982 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:00.648723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb80b3f-df47-4049-9540-6611f4512f71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:00.649399 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb80b3f-df47-4049-9540-6611f4512f71 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:00.649779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb80b3f-df47-4049-9540-6611f4512f71 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:00.649965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb80b3f-df47-4049-9540-6611f4512f71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:00.652178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ddb80b3f-df47-4049-9540-6611f4512f71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:00.664088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb80b3f-df47-4049-9540-6611f4512f71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:00.665687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb80b3f-df47-4049-9540-6611f4512f71 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:00.676071 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1205/2419] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:50:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:50:03.759541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a24127-686c-4176-8c98-5a9d7c8f439c service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:03.762251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-82a24127-686c-4176-8c98-5a9d7c8f439c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:03.828595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a24127-686c-4176-8c98-5a9d7c8f439c service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:03.833709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:03.834651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:03.834926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:03.835197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] PATH_INFO: `/v3/users/104a9c10adde46828fb8592a095e86df` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:03.836513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=104a9c10adde46828fb8592a095e86df)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:03.838534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:03.886734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '104a9c10adde46828fb8592a095e86df'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:03.887385 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '104a9c10adde46828fb8592a095e86df'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:03.900135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '104a9c10adde46828fb8592a095e86df'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:03.907660 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '104a9c10adde46828fb8592a095e86df'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:03.917059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '104a9c10adde46828fb8592a095e86df'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:03.923573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1215/2420] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:50:03 2022] DELETE /identity/v3/users/104a9c10adde46828fb8592a095e86df => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:50:04.125689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb80b3f-df47-4049-9540-6611f4512f71 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:04.136901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb80b3f-df47-4049-9540-6611f4512f71 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:04.194296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:04.195220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:04.195540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:04.195818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] PATH_INFO: `/v3/projects/10b8c7816f7f467ebec2809ec0a3fe7b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:04.197089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b8c7816f7f467ebec2809ec0a3fe7b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:04.199092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:04.200630 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:04.246456 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b8c7816f7f467ebec2809ec0a3fe7b', 'request_id': 'req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.252537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b8c7816f7f467ebec2809ec0a3fe7b', 'request_id': 'req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.258409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b8c7816f7f467ebec2809ec0a3fe7b', 'request_id': 'req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.274303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] The token cache is being invalidate because project 10b8c7816f7f467ebec2809ec0a3fe7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:50:04.274598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.275247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.276904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1206/2421] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:50:04 2022] DELETE /identity/v3/projects/10b8c7816f7f467ebec2809ec0a3fe7b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:50:04.281247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:04.282706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:04.296472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d17657-65ec-4938-a331-d6033e4892ca admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:04.301387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:04.358535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:04.359491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:04.359776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:04.360054 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] PATH_INFO: `/v3/users/60e4ad98be134148b8ae378ed4f84425` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:04.361387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e4ad98be134148b8ae378ed4f84425)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:04.363359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:04.410658 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e4ad98be134148b8ae378ed4f84425'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.411315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e4ad98be134148b8ae378ed4f84425'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.424034 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e4ad98be134148b8ae378ed4f84425'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.431233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e4ad98be134148b8ae378ed4f84425'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.440562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e4ad98be134148b8ae378ed4f84425'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.447276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1216/2422] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:50:04 2022] DELETE /identity/v3/users/60e4ad98be134148b8ae378ed4f84425 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:50:04.634354 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:04.636163 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:04.649769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e8c12-3d6a-40ae-8e4d-ba1ce3732204 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:04.711404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:04.712320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:04.712609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:04.712891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] PATH_INFO: `/v3/projects/5f17a3c1e2e740279814ec541d2b952e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:04.714119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f17a3c1e2e740279814ec541d2b952e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:04.716108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:04.766460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f17a3c1e2e740279814ec541d2b952e', 'request_id': 'req-f41c4dc5-cd43-45a3-997f-663c0b2417ac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.772904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f17a3c1e2e740279814ec541d2b952e', 'request_id': 'req-f41c4dc5-cd43-45a3-997f-663c0b2417ac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.779049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f17a3c1e2e740279814ec541d2b952e', 'request_id': 'req-f41c4dc5-cd43-45a3-997f-663c0b2417ac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.792586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] The token cache is being invalidate because project 5f17a3c1e2e740279814ec541d2b952e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:50:04.792898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.793558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f41c4dc5-cd43-45a3-997f-663c0b2417ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:50:04.795141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1207/2423] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:50:04 2022] DELETE /identity/v3/projects/5f17a3c1e2e740279814ec541d2b952e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:50:14.444248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:14.789724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=75,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:36.102438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:36.104063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:36.175167 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c5419-c1f2-4760-96f7-1f10491de28f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:36.235560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:36.236230 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:36.236519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:36.236808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:36.238690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:36.249875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:36.251496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:36.261321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1217/2424] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:50:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:50:38.307218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96a233-f7d4-4c86-9f80-d4fddd4f510b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:38.307765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96a233-f7d4-4c86-9f80-d4fddd4f510b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:38.307874 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96a233-f7d4-4c86-9f80-d4fddd4f510b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:38.326236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dc96a233-f7d4-4c86-9f80-d4fddd4f510b None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:38.327503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dc96a233-f7d4-4c86-9f80-d4fddd4f510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:38.390852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1208/2425] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:50:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:50:38.406286 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:38.407852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:38.481486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d8b00-e967-4e55-92c8-90c8bb057a7e service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:38.488085 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2145029-c77f-48b8-a721-c703db3f4af7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:38.488807 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2145029-c77f-48b8-a721-c703db3f4af7 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:38.489063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2145029-c77f-48b8-a721-c703db3f4af7 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:38.489332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2145029-c77f-48b8-a721-c703db3f4af7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:38.500203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2145029-c77f-48b8-a721-c703db3f4af7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:38.501873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2145029-c77f-48b8-a721-c703db3f4af7 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:38.510565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1218/2426] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:50:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:50:48.379506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=25 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:48.507074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:54.259635 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96a233-f7d4-4c86-9f80-d4fddd4f510b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:54.261124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dc96a233-f7d4-4c86-9f80-d4fddd4f510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:54.326392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96a233-f7d4-4c86-9f80-d4fddd4f510b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:54.330948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12135b-9671-47f0-aefb-03d9a19c33a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc21faf2237d405bba03e5309d93629f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:54.331214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12135b-9671-47f0-aefb-03d9a19c33a1 service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:54.331548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12135b-9671-47f0-aefb-03d9a19c33a1 service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:54.331822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12135b-9671-47f0-aefb-03d9a19c33a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:54.334222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3b12135b-9671-47f0-aefb-03d9a19c33a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:54.429743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12135b-9671-47f0-aefb-03d9a19c33a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:54.431201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12135b-9671-47f0-aefb-03d9a19c33a1 service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:54.451659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1209/2427] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:50:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:50:54.524174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2145029-c77f-48b8-a721-c703db3f4af7 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:54.525780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a2145029-c77f-48b8-a721-c703db3f4af7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:54.607731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2145029-c77f-48b8-a721-c703db3f4af7 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:54.611380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d2e80-b2a5-4f84-9eb1-40ce5ebfc852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:54.612062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d2e80-b2a5-4f84-9eb1-40ce5ebfc852 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:54.612346 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d2e80-b2a5-4f84-9eb1-40ce5ebfc852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:54.612783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d2e80-b2a5-4f84-9eb1-40ce5ebfc852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:54.614490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-883d2e80-b2a5-4f84-9eb1-40ce5ebfc852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:54.624565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d2e80-b2a5-4f84-9eb1-40ce5ebfc852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:54.625525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d2e80-b2a5-4f84-9eb1-40ce5ebfc852 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:54.633337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1219/2428] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:50:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:50:54.709284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12135b-9671-47f0-aefb-03d9a19c33a1 service glance] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:54.719304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12135b-9671-47f0-aefb-03d9a19c33a1 service glance] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:54.723299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f5407-2e65-4229-a48b-6bc637f74d37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:54.723945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f5407-2e65-4229-a48b-6bc637f74d37 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:54.724234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f5407-2e65-4229-a48b-6bc637f74d37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:54.724542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f5407-2e65-4229-a48b-6bc637f74d37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:54.726666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-872f5407-2e65-4229-a48b-6bc637f74d37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:54.740139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f5407-2e65-4229-a48b-6bc637f74d37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:54.741573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f5407-2e65-4229-a48b-6bc637f74d37 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:54.752445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1210/2429] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:50:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:50:56.789102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cae2cd-f859-47c1-8409-46b44906681c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.789102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cae2cd-f859-47c1-8409-46b44906681c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.789102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cae2cd-f859-47c1-8409-46b44906681c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.804086 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-76cae2cd-f859-47c1-8409-46b44906681c None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:56.805299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-76cae2cd-f859-47c1-8409-46b44906681c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:56.812785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1220/2430] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:50:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:50:56.834561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f5407-2e65-4229-a48b-6bc637f74d37 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:56.846501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f5407-2e65-4229-a48b-6bc637f74d37 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:56.852547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd8370-c972-43a3-8089-953098c65f4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:56.853200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd8370-c972-43a3-8089-953098c65f4f service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.853484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd8370-c972-43a3-8089-953098c65f4f service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.853761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd8370-c972-43a3-8089-953098c65f4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.863046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd8370-c972-43a3-8089-953098c65f4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:56.864705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd8370-c972-43a3-8089-953098c65f4f service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:56.873021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1211/2431] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:50:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:51:06.799038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=38 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:06.869659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=75 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:16.818597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fad0c-abd3-4590-b7e0-e62715d5bf3d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:16.819056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fad0c-abd3-4590-b7e0-e62715d5bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:16.819165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fad0c-abd3-4590-b7e0-e62715d5bf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.837243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-408fad0c-abd3-4590-b7e0-e62715d5bf3d None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:16.838452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-408fad0c-abd3-4590-b7e0-e62715d5bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:16.846178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1221/2432] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 13 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:16.858580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd8370-c972-43a3-8089-953098c65f4f service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:16.869705 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd8370-c972-43a3-8089-953098c65f4f service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:16.881712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c27435-d79e-481b-9a0c-fd82e1955107 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:16.882507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c27435-d79e-481b-9a0c-fd82e1955107 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:16.882663 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c27435-d79e-481b-9a0c-fd82e1955107 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:16.882943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c27435-d79e-481b-9a0c-fd82e1955107 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.892408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c27435-d79e-481b-9a0c-fd82e1955107 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:16.894039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c27435-d79e-481b-9a0c-fd82e1955107 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:16.902336 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1212/2433] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:51:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:51:19.498738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fad0c-abd3-4590-b7e0-e62715d5bf3d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:19.500574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-408fad0c-abd3-4590-b7e0-e62715d5bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:19.512962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fad0c-abd3-4590-b7e0-e62715d5bf3d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:19.517566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60381a21-3fe0-4d9d-a706-06c51237d970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:19.518224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60381a21-3fe0-4d9d-a706-06c51237d970 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:19.518510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60381a21-3fe0-4d9d-a706-06c51237d970 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:19.518784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60381a21-3fe0-4d9d-a706-06c51237d970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:19.520979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-60381a21-3fe0-4d9d-a706-06c51237d970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:19.585753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60381a21-3fe0-4d9d-a706-06c51237d970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:19.587211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60381a21-3fe0-4d9d-a706-06c51237d970 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:19.597076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1222/2434] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:22.783924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c27435-d79e-481b-9a0c-fd82e1955107 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:22.785697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f3c27435-d79e-481b-9a0c-fd82e1955107 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:22.801287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c27435-d79e-481b-9a0c-fd82e1955107 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:22.806724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:22.807611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:22.807891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:22.808165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] PATH_INFO: `/v3/users/bf4d42cf5bec422b96f421b98c84491d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:22.957412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4d42cf5bec422b96f421b98c84491d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:22.959375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.012917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4d42cf5bec422b96f421b98c84491d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.014059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4d42cf5bec422b96f421b98c84491d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.027262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4d42cf5bec422b96f421b98c84491d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.034124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4d42cf5bec422b96f421b98c84491d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.034872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-60533626-b3d7-4d3e-bb9b-e8464ee6e499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4d42cf5bec422b96f421b98c84491d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.049970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1213/2435] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:22 2022] DELETE /identity/v3/users/bf4d42cf5bec422b96f421b98c84491d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:23.228254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60381a21-3fe0-4d9d-a706-06c51237d970 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:23.243152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60381a21-3fe0-4d9d-a706-06c51237d970 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:23.304231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.305115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.305408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.305691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] PATH_INFO: `/v3/projects/3bb5a0bd02c6452597888d17874f760c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.316748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb5a0bd02c6452597888d17874f760c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.318710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.365415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb5a0bd02c6452597888d17874f760c', 'request_id': 'req-8cc55604-0cac-43b6-aa0d-1b43b3decc73'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.373973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb5a0bd02c6452597888d17874f760c', 'request_id': 'req-8cc55604-0cac-43b6-aa0d-1b43b3decc73'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.380089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb5a0bd02c6452597888d17874f760c', 'request_id': 'req-8cc55604-0cac-43b6-aa0d-1b43b3decc73'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.391768 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] The token cache is being invalidate because project 3bb5a0bd02c6452597888d17874f760c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:23.392066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.392797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8cc55604-0cac-43b6-aa0d-1b43b3decc73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:23.394518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1223/2436] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:23 2022] DELETE /identity/v3/projects/3bb5a0bd02c6452597888d17874f760c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:23.399846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa966fc4-81f0-4d30-8441-09272da4de08 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.400146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa966fc4-81f0-4d30-8441-09272da4de08 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.400426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa966fc4-81f0-4d30-8441-09272da4de08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.448216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-aa966fc4-81f0-4d30-8441-09272da4de08 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:23.449457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aa966fc4-81f0-4d30-8441-09272da4de08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:23.512988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1214/2437] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:23.521091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089104d-4931-45fd-9211-076eba29f228 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.521388 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089104d-4931-45fd-9211-076eba29f228 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.521672 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089104d-4931-45fd-9211-076eba29f228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.540517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f089104d-4931-45fd-9211-076eba29f228 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:23.541747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f089104d-4931-45fd-9211-076eba29f228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:23.550572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1224/2438] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:23.554845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa966fc4-81f0-4d30-8441-09272da4de08 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:23.565256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa966fc4-81f0-4d30-8441-09272da4de08 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:23.571173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15011a1-1181-4e1f-9aa2-7ad24e942ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.571875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15011a1-1181-4e1f-9aa2-7ad24e942ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.572161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15011a1-1181-4e1f-9aa2-7ad24e942ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.572447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15011a1-1181-4e1f-9aa2-7ad24e942ad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.573185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15011a1-1181-4e1f-9aa2-7ad24e942ad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:23.573499 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15011a1-1181-4e1f-9aa2-7ad24e942ad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.575132 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15011a1-1181-4e1f-9aa2-7ad24e942ad6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.587520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1215/2439] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:23.591830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089104d-4931-45fd-9211-076eba29f228 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:23.596246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:23.602312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089104d-4931-45fd-9211-076eba29f228 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:23.609121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf63ba-cf52-48fc-872e-f667689bee2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.609928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf63ba-cf52-48fc-872e-f667689bee2d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.610243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf63ba-cf52-48fc-872e-f667689bee2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.610530 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf63ba-cf52-48fc-872e-f667689bee2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.611269 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf63ba-cf52-48fc-872e-f667689bee2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.613746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf63ba-cf52-48fc-872e-f667689bee2d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.651721 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1225/2440] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:23 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:23.656728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15011a1-1181-4e1f-9aa2-7ad24e942ad6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:23.672669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15011a1-1181-4e1f-9aa2-7ad24e942ad6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:23.681721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:23.733552 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606f87-d5d7-4586-b631-7bb8cffa13d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.734305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606f87-d5d7-4586-b631-7bb8cffa13d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.734590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606f87-d5d7-4586-b631-7bb8cffa13d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.734898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606f87-d5d7-4586-b631-7bb8cffa13d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.735805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4606f87-d5d7-4586-b631-7bb8cffa13d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.737764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4606f87-d5d7-4586-b631-7bb8cffa13d9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.759975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1216/2441] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:23 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:23.764371 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf63ba-cf52-48fc-872e-f667689bee2d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:23.775118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf63ba-cf52-48fc-872e-f667689bee2d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:23.780917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda847d9-0a8e-454a-b5d6-85bc53ef68f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.781620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda847d9-0a8e-454a-b5d6-85bc53ef68f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.781876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda847d9-0a8e-454a-b5d6-85bc53ef68f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.782150 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda847d9-0a8e-454a-b5d6-85bc53ef68f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.782891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda847d9-0a8e-454a-b5d6-85bc53ef68f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.784559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda847d9-0a8e-454a-b5d6-85bc53ef68f4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.795126 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1226/2442] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:23.799792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606f87-d5d7-4586-b631-7bb8cffa13d9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:23.810305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606f87-d5d7-4586-b631-7bb8cffa13d9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:23.816774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04350d-0795-4eb7-a9fd-4424b8b5c981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.817449 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04350d-0795-4eb7-a9fd-4424b8b5c981 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.817732 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04350d-0795-4eb7-a9fd-4424b8b5c981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.818006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04350d-0795-4eb7-a9fd-4424b8b5c981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.818739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04350d-0795-4eb7-a9fd-4424b8b5c981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.820425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04350d-0795-4eb7-a9fd-4424b8b5c981 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.830875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1217/2443] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:23.835109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda847d9-0a8e-454a-b5d6-85bc53ef68f4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:23.846000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda847d9-0a8e-454a-b5d6-85bc53ef68f4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:23.851978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd4065-7f15-4d52-963f-2902f9c391c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.852885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd4065-7f15-4d52-963f-2902f9c391c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.853159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd4065-7f15-4d52-963f-2902f9c391c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.853435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd4065-7f15-4d52-963f-2902f9c391c8 admin admin] PATH_INFO: `/v3/projects/5ac3fe132bc242689f13a108271b8e9a/users/d5389c1b2b9c4eb4996b28a5dd2e4dbc/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.870503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd4065-7f15-4d52-963f-2902f9c391c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac3fe132bc242689f13a108271b8e9a, user_id=d5389c1b2b9c4eb4996b28a5dd2e4dbc, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.873254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd4065-7f15-4d52-963f-2902f9c391c8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.888222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1227/2444] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:23 2022] PUT /identity/v3/projects/5ac3fe132bc242689f13a108271b8e9a/users/d5389c1b2b9c4eb4996b28a5dd2e4dbc/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:23.898558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51618fc4-371f-4fdc-9606-4d33781fc716 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.898872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51618fc4-371f-4fdc-9606-4d33781fc716 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.899173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51618fc4-371f-4fdc-9606-4d33781fc716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.927726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-51618fc4-371f-4fdc-9606-4d33781fc716 None None] MFA Rules not processed for user `d5389c1b2b9c4eb4996b28a5dd2e4dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:23.928913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-51618fc4-371f-4fdc-9606-4d33781fc716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:23.997179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1218/2445] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:51:24.001954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd4065-7f15-4d52-963f-2902f9c391c8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.065317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd4065-7f15-4d52-963f-2902f9c391c8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.070475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ff36a-53b4-46eb-838c-4615a7f374f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.071157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ff36a-53b4-46eb-838c-4615a7f374f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.071441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ff36a-53b4-46eb-838c-4615a7f374f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.071876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ff36a-53b4-46eb-838c-4615a7f374f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.073162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ff36a-53b4-46eb-838c-4615a7f374f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.075692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ff36a-53b4-46eb-838c-4615a7f374f2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.114112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1228/2446] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:24 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:24.118627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51618fc4-371f-4fdc-9606-4d33781fc716 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.130194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51618fc4-371f-4fdc-9606-4d33781fc716 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.189832 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af83484-d146-4d9d-bb2c-b5c41c0c2a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.190542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af83484-d146-4d9d-bb2c-b5c41c0c2a71 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.190828 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af83484-d146-4d9d-bb2c-b5c41c0c2a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.191101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af83484-d146-4d9d-bb2c-b5c41c0c2a71 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.191845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af83484-d146-4d9d-bb2c-b5c41c0c2a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.193797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af83484-d146-4d9d-bb2c-b5c41c0c2a71 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.215512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1219/2447] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:24 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:24.219903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ff36a-53b4-46eb-838c-4615a7f374f2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.230738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ff36a-53b4-46eb-838c-4615a7f374f2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.236186 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1f29b-2373-4d65-a064-438c73365ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.236905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1f29b-2373-4d65-a064-438c73365ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.237177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1f29b-2373-4d65-a064-438c73365ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.237490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1f29b-2373-4d65-a064-438c73365ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.238222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1f29b-2373-4d65-a064-438c73365ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.239948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1f29b-2373-4d65-a064-438c73365ff2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.250973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1229/2448] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:51:24.255674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af83484-d146-4d9d-bb2c-b5c41c0c2a71 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.266603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af83484-d146-4d9d-bb2c-b5c41c0c2a71 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.272066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7907b-68cb-4a28-8866-e6e1ca5d7c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.272948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7907b-68cb-4a28-8866-e6e1ca5d7c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.273234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7907b-68cb-4a28-8866-e6e1ca5d7c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.273509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7907b-68cb-4a28-8866-e6e1ca5d7c7d admin admin] PATH_INFO: `/v3/projects/e1bead8256e94731a220a1a581301b40/users/d0bb5f1f676b4b98a0ef9c7554280796/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.276903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:24.290134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7907b-68cb-4a28-8866-e6e1ca5d7c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bead8256e94731a220a1a581301b40, user_id=d0bb5f1f676b4b98a0ef9c7554280796, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.292814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7907b-68cb-4a28-8866-e6e1ca5d7c7d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.306977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1220/2449] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:24 2022] PUT /identity/v3/projects/e1bead8256e94731a220a1a581301b40/users/d0bb5f1f676b4b98a0ef9c7554280796/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:24.312017 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1f29b-2373-4d65-a064-438c73365ff2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.324188 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1f29b-2373-4d65-a064-438c73365ff2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.389231 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594871b9-67fe-495e-9af6-feb3333849f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.389940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594871b9-67fe-495e-9af6-feb3333849f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.390246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594871b9-67fe-495e-9af6-feb3333849f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.390599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594871b9-67fe-495e-9af6-feb3333849f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.391337 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594871b9-67fe-495e-9af6-feb3333849f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.392990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594871b9-67fe-495e-9af6-feb3333849f2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.406778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1230/2450] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:24.408177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7907b-68cb-4a28-8866-e6e1ca5d7c7d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.418468 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7907b-68cb-4a28-8866-e6e1ca5d7c7d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.423563 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bae9b4-49ad-435e-9b8d-c2dfc9dd5bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.424457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bae9b4-49ad-435e-9b8d-c2dfc9dd5bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.424739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bae9b4-49ad-435e-9b8d-c2dfc9dd5bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.425011 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bae9b4-49ad-435e-9b8d-c2dfc9dd5bf3 admin admin] PATH_INFO: `/v3/projects/e1bead8256e94731a220a1a581301b40/users/d0bb5f1f676b4b98a0ef9c7554280796/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.426959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bae9b4-49ad-435e-9b8d-c2dfc9dd5bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bead8256e94731a220a1a581301b40, user_id=d0bb5f1f676b4b98a0ef9c7554280796, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.429629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bae9b4-49ad-435e-9b8d-c2dfc9dd5bf3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.440644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1221/2451] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:24 2022] PUT /identity/v3/projects/e1bead8256e94731a220a1a581301b40/users/d0bb5f1f676b4b98a0ef9c7554280796/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:24.452087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec4203-3de2-4a20-9fe0-fb074fd0b9e2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.452379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec4203-3de2-4a20-9fe0-fb074fd0b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.452659 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec4203-3de2-4a20-9fe0-fb074fd0b9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.466857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:24.482525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c9ec4203-3de2-4a20-9fe0-fb074fd0b9e2 None None] MFA Rules not processed for user `d0bb5f1f676b4b98a0ef9c7554280796`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:24.483806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c9ec4203-3de2-4a20-9fe0-fb074fd0b9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:24.551909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1231/2452] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:24.564146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bae9b4-49ad-435e-9b8d-c2dfc9dd5bf3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.566014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-76bae9b4-49ad-435e-9b8d-c2dfc9dd5bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:24.646053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bae9b4-49ad-435e-9b8d-c2dfc9dd5bf3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.650669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8737b-67ee-435c-92c7-6f67851e78f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.651320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8737b-67ee-435c-92c7-6f67851e78f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.651742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8737b-67ee-435c-92c7-6f67851e78f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.651989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8737b-67ee-435c-92c7-6f67851e78f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.664863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8737b-67ee-435c-92c7-6f67851e78f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.666288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8737b-67ee-435c-92c7-6f67851e78f4 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.676077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1222/2453] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:24.761967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec4203-3de2-4a20-9fe0-fb074fd0b9e2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.772471 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec4203-3de2-4a20-9fe0-fb074fd0b9e2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.776963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ca237-9fb9-4b65-83c7-9073aa37bcf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.777613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ca237-9fb9-4b65-83c7-9073aa37bcf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.777892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ca237-9fb9-4b65-83c7-9073aa37bcf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.778169 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ca237-9fb9-4b65-83c7-9073aa37bcf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.840008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ca237-9fb9-4b65-83c7-9073aa37bcf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.841381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ca237-9fb9-4b65-83c7-9073aa37bcf7 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.850432 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1232/2454] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:25.302460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8737b-67ee-435c-92c7-6f67851e78f4 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:25.311698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8737b-67ee-435c-92c7-6f67851e78f4 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:25.315863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:25.316496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:25.316774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:25.317046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.319300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:25.391327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:25.392697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:25.403961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1223/2455] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:28.063720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ca237-9fb9-4b65-83c7-9073aa37bcf7 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:28.065476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-de8ca237-9fb9-4b65-83c7-9073aa37bcf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:28.126244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ca237-9fb9-4b65-83c7-9073aa37bcf7 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:28.131159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2598d6b-183f-4649-a779-bf4c8f1759d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:28.131854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598d6b-183f-4649-a779-bf4c8f1759d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:28.132146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598d6b-183f-4649-a779-bf4c8f1759d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:28.132425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598d6b-183f-4649-a779-bf4c8f1759d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:28.134590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2598d6b-183f-4649-a779-bf4c8f1759d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:28.149094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2598d6b-183f-4649-a779-bf4c8f1759d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:28.150548 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2598d6b-183f-4649-a779-bf4c8f1759d0 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:28.162077 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1233/2456] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:29.318839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:29.320478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:29.334495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639510e3-75e0-4715-b70f-90dd3304acb7 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:29.341108 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f3128-04e3-4612-84fa-1d26168a85d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:29.341765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f3128-04e3-4612-84fa-1d26168a85d2 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.342081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f3128-04e3-4612-84fa-1d26168a85d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.342360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f3128-04e3-4612-84fa-1d26168a85d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.344578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ec8f3128-04e3-4612-84fa-1d26168a85d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:29.383951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:29.404683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f3128-04e3-4612-84fa-1d26168a85d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:29.406288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f3128-04e3-4612-84fa-1d26168a85d2 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:29.416340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1224/2457] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:36.027238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2598d6b-183f-4649-a779-bf4c8f1759d0 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.110582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:36.152632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2598d6b-183f-4649-a779-bf4c8f1759d0 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.159380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.161481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.161998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.162407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] PATH_INFO: `/v3/users/d5389c1b2b9c4eb4996b28a5dd2e4dbc` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.164068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5389c1b2b9c4eb4996b28a5dd2e4dbc)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:36.167136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:36.255562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5389c1b2b9c4eb4996b28a5dd2e4dbc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.256535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5389c1b2b9c4eb4996b28a5dd2e4dbc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.272709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5389c1b2b9c4eb4996b28a5dd2e4dbc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.283339 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5389c1b2b9c4eb4996b28a5dd2e4dbc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.295376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5389c1b2b9c4eb4996b28a5dd2e4dbc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.303733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1234/2458] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:36 2022] DELETE /identity/v3/users/d5389c1b2b9c4eb4996b28a5dd2e4dbc => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:36.310315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39451e-a131-4897-9339-0cf0826514f0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.310831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39451e-a131-4897-9339-0cf0826514f0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.310942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39451e-a131-4897-9339-0cf0826514f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.332577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ab39451e-a131-4897-9339-0cf0826514f0 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:36.333712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ab39451e-a131-4897-9339-0cf0826514f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:36.393538 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1225/2459] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:36.403740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.405214 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:36.468152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72a60-487d-4d39-b5e4-fc945d856d48 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.472568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5c86f-53ae-40a3-8efe-01217398d064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.473324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5c86f-53ae-40a3-8efe-01217398d064 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.473610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5c86f-53ae-40a3-8efe-01217398d064 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.474030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5c86f-53ae-40a3-8efe-01217398d064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.486151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5c86f-53ae-40a3-8efe-01217398d064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:36.487876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5c86f-53ae-40a3-8efe-01217398d064 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:36.498141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1235/2460] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:36.782159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39451e-a131-4897-9339-0cf0826514f0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.783978 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ab39451e-a131-4897-9339-0cf0826514f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:36.797107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39451e-a131-4897-9339-0cf0826514f0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.802950 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.804035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.804325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.804650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] PATH_INFO: `/v3/projects/5ac3fe132bc242689f13a108271b8e9a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.806101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac3fe132bc242689f13a108271b8e9a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:36.808082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:36.864920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac3fe132bc242689f13a108271b8e9a', 'request_id': 'req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.871504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac3fe132bc242689f13a108271b8e9a', 'request_id': 'req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.880164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac3fe132bc242689f13a108271b8e9a', 'request_id': 'req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.893470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] The token cache is being invalidate because project 5ac3fe132bc242689f13a108271b8e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:36.893789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.894560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:36.896401 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1226/2461] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:36 2022] DELETE /identity/v3/projects/5ac3fe132bc242689f13a108271b8e9a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:36.901427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5c86f-53ae-40a3-8efe-01217398d064 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.911890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5c86f-53ae-40a3-8efe-01217398d064 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.970550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:36.980686 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.981822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.982277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.982636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] PATH_INFO: `/v3/users/d0bb5f1f676b4b98a0ef9c7554280796` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.984710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0bb5f1f676b4b98a0ef9c7554280796)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:36.987447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:37.041057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0bb5f1f676b4b98a0ef9c7554280796'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.041777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0bb5f1f676b4b98a0ef9c7554280796'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.055667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0bb5f1f676b4b98a0ef9c7554280796'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.066811 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0bb5f1f676b4b98a0ef9c7554280796'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.074847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-116bbea3-6bba-49de-bab0-c1bbf19d9ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0bb5f1f676b4b98a0ef9c7554280796'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.080998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1236/2462] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:36 2022] DELETE /identity/v3/users/d0bb5f1f676b4b98a0ef9c7554280796 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:37.363758 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:37.366773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:37.381164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a8d1e-0aeb-46f0-ad25-82ce1d7ec458 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:37.455767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:37.456714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.457091 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.457411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] PATH_INFO: `/v3/projects/e1bead8256e94731a220a1a581301b40` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.458671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bead8256e94731a220a1a581301b40)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:37.460669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:37.463112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:37.513154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bead8256e94731a220a1a581301b40', 'request_id': 'req-33961900-4f2a-4b8c-9093-8676ad4f78cd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.519348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bead8256e94731a220a1a581301b40', 'request_id': 'req-33961900-4f2a-4b8c-9093-8676ad4f78cd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.532069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bead8256e94731a220a1a581301b40', 'request_id': 'req-33961900-4f2a-4b8c-9093-8676ad4f78cd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.547140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] The token cache is being invalidate because project e1bead8256e94731a220a1a581301b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:37.547434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.548082 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33961900-4f2a-4b8c-9093-8676ad4f78cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:37.549639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1227/2463] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:37 2022] DELETE /identity/v3/projects/e1bead8256e94731a220a1a581301b40 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:37.554645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dcc0e-ded1-4bea-ab5a-dfa70a81b024 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.554942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dcc0e-ded1-4bea-ab5a-dfa70a81b024 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.555224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dcc0e-ded1-4bea-ab5a-dfa70a81b024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.574444 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a01dcc0e-ded1-4bea-ab5a-dfa70a81b024 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:37.575688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a01dcc0e-ded1-4bea-ab5a-dfa70a81b024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:37.638892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1237/2464] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:37.647136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf8920-b0c3-4916-9b3f-bdcbc71a1fd4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.647430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf8920-b0c3-4916-9b3f-bdcbc71a1fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.647741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf8920-b0c3-4916-9b3f-bdcbc71a1fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.666576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cbbf8920-b0c3-4916-9b3f-bdcbc71a1fd4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:37.667812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cbbf8920-b0c3-4916-9b3f-bdcbc71a1fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:37.676330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1228/2465] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:37.680638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dcc0e-ded1-4bea-ab5a-dfa70a81b024 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:37.691316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dcc0e-ded1-4bea-ab5a-dfa70a81b024 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:37.696927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19192fa3-b76b-41a2-bd11-ddc39c55a2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:37.697603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19192fa3-b76b-41a2-bd11-ddc39c55a2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.697886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19192fa3-b76b-41a2-bd11-ddc39c55a2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.698162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19192fa3-b76b-41a2-bd11-ddc39c55a2a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.698909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19192fa3-b76b-41a2-bd11-ddc39c55a2a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:37.699215 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19192fa3-b76b-41a2-bd11-ddc39c55a2a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:37.700915 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19192fa3-b76b-41a2-bd11-ddc39c55a2a0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:37.713677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1238/2466] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:37.717970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf8920-b0c3-4916-9b3f-bdcbc71a1fd4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:37.727652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf8920-b0c3-4916-9b3f-bdcbc71a1fd4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:37.732669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92bfc1-2f60-437b-a82a-23477805a7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:37.733289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92bfc1-2f60-437b-a82a-23477805a7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.733576 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92bfc1-2f60-437b-a82a-23477805a7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.733856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92bfc1-2f60-437b-a82a-23477805a7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.734681 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92bfc1-2f60-437b-a82a-23477805a7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:37.736708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92bfc1-2f60-437b-a82a-23477805a7b4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:37.764922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1229/2467] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:37 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:37.769340 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19192fa3-b76b-41a2-bd11-ddc39c55a2a0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:37.780199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19192fa3-b76b-41a2-bd11-ddc39c55a2a0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:37.846253 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542501c0-51d5-42b6-b47c-d4abce0b0105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:37.847301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542501c0-51d5-42b6-b47c-d4abce0b0105 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.847856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542501c0-51d5-42b6-b47c-d4abce0b0105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.848264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542501c0-51d5-42b6-b47c-d4abce0b0105 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.849634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542501c0-51d5-42b6-b47c-d4abce0b0105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:37.853239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542501c0-51d5-42b6-b47c-d4abce0b0105 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:37.864991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:37.880631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1239/2468] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:37 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:37.884864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92bfc1-2f60-437b-a82a-23477805a7b4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:37.895691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92bfc1-2f60-437b-a82a-23477805a7b4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:37.901233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8309d-21c2-498d-bbba-e3cb99de2dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:37.901904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8309d-21c2-498d-bbba-e3cb99de2dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.902189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8309d-21c2-498d-bbba-e3cb99de2dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.902463 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8309d-21c2-498d-bbba-e3cb99de2dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.903192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8309d-21c2-498d-bbba-e3cb99de2dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:37.904850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8309d-21c2-498d-bbba-e3cb99de2dcd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:37.918211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1230/2469] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:37.924824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542501c0-51d5-42b6-b47c-d4abce0b0105 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:37.941971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542501c0-51d5-42b6-b47c-d4abce0b0105 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:37.948129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e1dca-f311-4787-90b0-b724f3a8e948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:37.948967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e1dca-f311-4787-90b0-b724f3a8e948 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.949254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e1dca-f311-4787-90b0-b724f3a8e948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.949536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e1dca-f311-4787-90b0-b724f3a8e948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.950233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e1dca-f311-4787-90b0-b724f3a8e948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:37.952015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e1dca-f311-4787-90b0-b724f3a8e948 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:37.963199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1240/2470] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:37.967487 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8309d-21c2-498d-bbba-e3cb99de2dcd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:37.978612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8309d-21c2-498d-bbba-e3cb99de2dcd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:37.983178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc745ed-a7b6-4cb0-a99b-85bad46f4147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:37.984050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc745ed-a7b6-4cb0-a99b-85bad46f4147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.984331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc745ed-a7b6-4cb0-a99b-85bad46f4147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.984607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc745ed-a7b6-4cb0-a99b-85bad46f4147 admin admin] PATH_INFO: `/v3/projects/52f134e182954c3a8440b2bc6696e2da/users/577e30bf83d2471392b170b45d8da0a9/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.000439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc745ed-a7b6-4cb0-a99b-85bad46f4147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f134e182954c3a8440b2bc6696e2da, user_id=577e30bf83d2471392b170b45d8da0a9, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.003183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc745ed-a7b6-4cb0-a99b-85bad46f4147 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.017246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1231/2471] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:37 2022] PUT /identity/v3/projects/52f134e182954c3a8440b2bc6696e2da/users/577e30bf83d2471392b170b45d8da0a9/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:38.027712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0ce76-386b-4a93-b5fe-9ce60390ec77 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.028102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0ce76-386b-4a93-b5fe-9ce60390ec77 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.028451 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0ce76-386b-4a93-b5fe-9ce60390ec77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.062893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-52e0ce76-386b-4a93-b5fe-9ce60390ec77 None None] MFA Rules not processed for user `577e30bf83d2471392b170b45d8da0a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:38.064172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-52e0ce76-386b-4a93-b5fe-9ce60390ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:38.136315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1241/2472] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:38.140707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc745ed-a7b6-4cb0-a99b-85bad46f4147 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.212801 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc745ed-a7b6-4cb0-a99b-85bad46f4147 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.215123 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0ce76-386b-4a93-b5fe-9ce60390ec77 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.216640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-52e0ce76-386b-4a93-b5fe-9ce60390ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:38.217490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea28d-ddc4-43c2-946c-08393061c500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.218165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea28d-ddc4-43c2-946c-08393061c500 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.218451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea28d-ddc4-43c2-946c-08393061c500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.218728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea28d-ddc4-43c2-946c-08393061c500 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.219425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dea28d-ddc4-43c2-946c-08393061c500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.221395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dea28d-ddc4-43c2-946c-08393061c500 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.252317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1232/2473] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:38 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:38.253304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea28d-ddc4-43c2-946c-08393061c500 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.272204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:38.283213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0ce76-386b-4a93-b5fe-9ce60390ec77 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.325708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea28d-ddc4-43c2-946c-08393061c500 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.330319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deb97a-02d8-4b91-871f-75d7591dacb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.331040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb97a-02d8-4b91-871f-75d7591dacb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.331324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb97a-02d8-4b91-871f-75d7591dacb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.331631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb97a-02d8-4b91-871f-75d7591dacb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.332317 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95deb97a-02d8-4b91-871f-75d7591dacb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.333260 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe598f5-5210-46e0-945f-d3fc375daddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.333937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe598f5-5210-46e0-945f-d3fc375daddd service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.334222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe598f5-5210-46e0-945f-d3fc375daddd service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.334330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95deb97a-02d8-4b91-871f-75d7591dacb9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.334500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe598f5-5210-46e0-945f-d3fc375daddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.336405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dfe598f5-5210-46e0-945f-d3fc375daddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:38.348966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe598f5-5210-46e0-945f-d3fc375daddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.350470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe598f5-5210-46e0-945f-d3fc375daddd service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.355426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1233/2474] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:38 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:38.359362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deb97a-02d8-4b91-871f-75d7591dacb9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.364295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1242/2475] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:51:38.369577 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deb97a-02d8-4b91-871f-75d7591dacb9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.374530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dcd93f-cd67-4dc3-8ffc-889b253c950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.375198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcd93f-cd67-4dc3-8ffc-889b253c950a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.375535 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcd93f-cd67-4dc3-8ffc-889b253c950a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.375820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcd93f-cd67-4dc3-8ffc-889b253c950a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.376549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dcd93f-cd67-4dc3-8ffc-889b253c950a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.378548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dcd93f-cd67-4dc3-8ffc-889b253c950a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.388956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1234/2476] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:51:38.393029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe598f5-5210-46e0-945f-d3fc375daddd service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.402992 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe598f5-5210-46e0-945f-d3fc375daddd service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.407836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba513a-c5f4-4860-936d-68f9c0b6c0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.408701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba513a-c5f4-4860-936d-68f9c0b6c0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.408987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba513a-c5f4-4860-936d-68f9c0b6c0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.409262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba513a-c5f4-4860-936d-68f9c0b6c0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.409985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba513a-c5f4-4860-936d-68f9c0b6c0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.411622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba513a-c5f4-4860-936d-68f9c0b6c0c2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.421784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1243/2477] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:38.426154 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dcd93f-cd67-4dc3-8ffc-889b253c950a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.436962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dcd93f-cd67-4dc3-8ffc-889b253c950a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.442170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2a492-701a-4ce4-bc59-14dbab8a7c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.443072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2a492-701a-4ce4-bc59-14dbab8a7c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.443356 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2a492-701a-4ce4-bc59-14dbab8a7c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.443668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2a492-701a-4ce4-bc59-14dbab8a7c8c admin admin] PATH_INFO: `/v3/projects/b9a6ba4fded14525ac507be850bcec3b/users/ab894f59a56646af9495c008afd9001f/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.459404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2a492-701a-4ce4-bc59-14dbab8a7c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a6ba4fded14525ac507be850bcec3b, user_id=ab894f59a56646af9495c008afd9001f, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.462092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2a492-701a-4ce4-bc59-14dbab8a7c8c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.476659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1235/2478] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:38 2022] PUT /identity/v3/projects/b9a6ba4fded14525ac507be850bcec3b/users/ab894f59a56646af9495c008afd9001f/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:38.488304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544f443-d9a3-4973-bc4f-124858114484 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.488598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544f443-d9a3-4973-bc4f-124858114484 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.488893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544f443-d9a3-4973-bc4f-124858114484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.521851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e544f443-d9a3-4973-bc4f-124858114484 None None] MFA Rules not processed for user `ab894f59a56646af9495c008afd9001f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:38.523005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e544f443-d9a3-4973-bc4f-124858114484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:38.528281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:38.597783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1244/2479] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:38.602265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2a492-701a-4ce4-bc59-14dbab8a7c8c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.697230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2a492-701a-4ce4-bc59-14dbab8a7c8c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.709123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e39a1-2d32-4f26-92a6-67f9592276e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.710427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e39a1-2d32-4f26-92a6-67f9592276e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.710722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e39a1-2d32-4f26-92a6-67f9592276e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.711005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e39a1-2d32-4f26-92a6-67f9592276e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.711833 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e39a1-2d32-4f26-92a6-67f9592276e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.713769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e39a1-2d32-4f26-92a6-67f9592276e6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.747031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1236/2480] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:38 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:38.751142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544f443-d9a3-4973-bc4f-124858114484 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.762468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544f443-d9a3-4973-bc4f-124858114484 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.840725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135c7dc-bb1d-4a2f-9f70-69d0a79aabf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.841420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135c7dc-bb1d-4a2f-9f70-69d0a79aabf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.841710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135c7dc-bb1d-4a2f-9f70-69d0a79aabf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.841985 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135c7dc-bb1d-4a2f-9f70-69d0a79aabf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.842693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135c7dc-bb1d-4a2f-9f70-69d0a79aabf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.844673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135c7dc-bb1d-4a2f-9f70-69d0a79aabf2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.868932 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1245/2481] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:38 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:38.870198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e39a1-2d32-4f26-92a6-67f9592276e6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.881133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e39a1-2d32-4f26-92a6-67f9592276e6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.885793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6d34b-4798-4600-bd8e-37129b9f9e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.886500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6d34b-4798-4600-bd8e-37129b9f9e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.886789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6d34b-4798-4600-bd8e-37129b9f9e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.887063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6d34b-4798-4600-bd8e-37129b9f9e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.887850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6d34b-4798-4600-bd8e-37129b9f9e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.889591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6d34b-4798-4600-bd8e-37129b9f9e02 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.901261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1237/2482] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:38.905682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135c7dc-bb1d-4a2f-9f70-69d0a79aabf2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.918596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135c7dc-bb1d-4a2f-9f70-69d0a79aabf2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.923284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744ecf5-cfe0-4d25-b6bc-9277cef4583d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.924204 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744ecf5-cfe0-4d25-b6bc-9277cef4583d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.924519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744ecf5-cfe0-4d25-b6bc-9277cef4583d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.924801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744ecf5-cfe0-4d25-b6bc-9277cef4583d admin admin] PATH_INFO: `/v3/projects/de7779237e414aa1814f9169736087cc/users/e0f039a3eb6845f297c1b693a052dfb8/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.941206 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744ecf5-cfe0-4d25-b6bc-9277cef4583d admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7779237e414aa1814f9169736087cc, user_id=e0f039a3eb6845f297c1b693a052dfb8, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.944056 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744ecf5-cfe0-4d25-b6bc-9277cef4583d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.958838 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1246/2483] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:38 2022] PUT /identity/v3/projects/de7779237e414aa1814f9169736087cc/users/e0f039a3eb6845f297c1b693a052dfb8/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:38.962993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6d34b-4798-4600-bd8e-37129b9f9e02 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.973310 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6d34b-4798-4600-bd8e-37129b9f9e02 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:39.015392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:39.034543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e95fd-be23-426e-b56f-cfd864d4ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:39.035150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e95fd-be23-426e-b56f-cfd864d4ed16 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:39.035441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e95fd-be23-426e-b56f-cfd864d4ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:39.035743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e95fd-be23-426e-b56f-cfd864d4ed16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:39.036487 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e95fd-be23-426e-b56f-cfd864d4ed16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:39.038089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e95fd-be23-426e-b56f-cfd864d4ed16 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:39.048745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1238/2484] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:39.053479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744ecf5-cfe0-4d25-b6bc-9277cef4583d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:39.064567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744ecf5-cfe0-4d25-b6bc-9277cef4583d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:39.070099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e7bc4-a676-4ec0-b8e7-fb50e36b49ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:39.071020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e7bc4-a676-4ec0-b8e7-fb50e36b49ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:39.071300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e7bc4-a676-4ec0-b8e7-fb50e36b49ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:39.071602 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e7bc4-a676-4ec0-b8e7-fb50e36b49ba admin admin] PATH_INFO: `/v3/projects/de7779237e414aa1814f9169736087cc/users/e0f039a3eb6845f297c1b693a052dfb8/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:39.073618 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e7bc4-a676-4ec0-b8e7-fb50e36b49ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7779237e414aa1814f9169736087cc, user_id=e0f039a3eb6845f297c1b693a052dfb8, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:39.076531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e7bc4-a676-4ec0-b8e7-fb50e36b49ba admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:39.089917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1247/2485] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:39 2022] PUT /identity/v3/projects/de7779237e414aa1814f9169736087cc/users/e0f039a3eb6845f297c1b693a052dfb8/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:39.101080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9cc68-71a9-48b4-aa52-44c7964f3084 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:39.101437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9cc68-71a9-48b4-aa52-44c7964f3084 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:39.102089 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9cc68-71a9-48b4-aa52-44c7964f3084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:39.134155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3cd9cc68-71a9-48b4-aa52-44c7964f3084 None None] MFA Rules not processed for user `e0f039a3eb6845f297c1b693a052dfb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:39.135586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3cd9cc68-71a9-48b4-aa52-44c7964f3084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:39.207581 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1239/2486] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:39.218779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e7bc4-a676-4ec0-b8e7-fb50e36b49ba admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:39.220655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9d2e7bc4-a676-4ec0-b8e7-fb50e36b49ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:39.280076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e7bc4-a676-4ec0-b8e7-fb50e36b49ba admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:39.283707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043d200-a96a-4f48-8512-fa970cf5291a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:39.284400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043d200-a96a-4f48-8512-fa970cf5291a service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:39.284750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043d200-a96a-4f48-8512-fa970cf5291a service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:39.284963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043d200-a96a-4f48-8512-fa970cf5291a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:39.294587 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043d200-a96a-4f48-8512-fa970cf5291a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:39.297072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043d200-a96a-4f48-8512-fa970cf5291a service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:39.308141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1248/2487] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:39.862440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9cc68-71a9-48b4-aa52-44c7964f3084 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:39.872304 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9cc68-71a9-48b4-aa52-44c7964f3084 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:39.876260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c6294-8948-42a8-a2f7-ee8ad1c4955a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:39.876930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c6294-8948-42a8-a2f7-ee8ad1c4955a service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:39.877229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c6294-8948-42a8-a2f7-ee8ad1c4955a service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:39.877509 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c6294-8948-42a8-a2f7-ee8ad1c4955a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:39.948720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c6294-8948-42a8-a2f7-ee8ad1c4955a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:39.949209 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043d200-a96a-4f48-8512-fa970cf5291a service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:39.950410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c6294-8948-42a8-a2f7-ee8ad1c4955a service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:39.951221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2043d200-a96a-4f48-8512-fa970cf5291a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:39.958295 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:39.960331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1240/2488] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:40.020066 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043d200-a96a-4f48-8512-fa970cf5291a service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:40.024111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:40.024309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:40.024597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:40.024873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:40.027088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:40.129741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:40.131632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:40.154088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1249/2489] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:51:44.230496 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c6294-8948-42a8-a2f7-ee8ad1c4955a service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.232594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4c8c6294-8948-42a8-a2f7-ee8ad1c4955a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:44.251381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c6294-8948-42a8-a2f7-ee8ad1c4955a service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.259973 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.261369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.262043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.262043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] PATH_INFO: `/v3/users/0b22e96941dd414da4fa22ed4bd36984` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.285223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b22e96941dd414da4fa22ed4bd36984)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.287889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.332288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.333797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:44.349350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:44.351350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b22e96941dd414da4fa22ed4bd36984'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.352025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23c95e-76a2-42ef-8da5-edff5a7cebfd service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.352145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b22e96941dd414da4fa22ed4bd36984'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.358508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fc858-87c1-4d8d-8c0a-34451596d06e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.359200 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fc858-87c1-4d8d-8c0a-34451596d06e service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.359499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fc858-87c1-4d8d-8c0a-34451596d06e service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.359777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fc858-87c1-4d8d-8c0a-34451596d06e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.380889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1d8fc858-87c1-4d8d-8c0a-34451596d06e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:44.388649 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b22e96941dd414da4fa22ed4bd36984'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.398826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b22e96941dd414da4fa22ed4bd36984'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.399911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b22e96941dd414da4fa22ed4bd36984'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.425412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1241/2490] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:44 2022] DELETE /identity/v3/users/0b22e96941dd414da4fa22ed4bd36984 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:44.514553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fc858-87c1-4d8d-8c0a-34451596d06e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.516203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fc858-87c1-4d8d-8c0a-34451596d06e service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.533051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1250/2491] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:51:44.696948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.700469 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:44.812217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b845392-998a-46e4-a939-ebf23caed245 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.817847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.819475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.820019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.820520 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] PATH_INFO: `/v3/projects/d006993d66cb4976b40858d4373f0e94` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.849728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d006993d66cb4976b40858d4373f0e94)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.852855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.930135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd006993d66cb4976b40858d4373f0e94', 'request_id': 'req-20a68cff-0bf0-4d32-8ba3-84637b29267f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.937848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd006993d66cb4976b40858d4373f0e94', 'request_id': 'req-20a68cff-0bf0-4d32-8ba3-84637b29267f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.944032 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd006993d66cb4976b40858d4373f0e94', 'request_id': 'req-20a68cff-0bf0-4d32-8ba3-84637b29267f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.964410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] The token cache is being invalidate because project d006993d66cb4976b40858d4373f0e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:44.964782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.965612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-20a68cff-0bf0-4d32-8ba3-84637b29267f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:44.970887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1242/2492] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:44 2022] DELETE /identity/v3/projects/d006993d66cb4976b40858d4373f0e94 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:44.972836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd256fd-3c3a-4dd8-a2f3-b9f484ef5156 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.973797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd256fd-3c3a-4dd8-a2f3-b9f484ef5156 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.974297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd256fd-3c3a-4dd8-a2f3-b9f484ef5156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.994556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-efd256fd-3c3a-4dd8-a2f3-b9f484ef5156 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:44.995850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-efd256fd-3c3a-4dd8-a2f3-b9f484ef5156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:45.045393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:45.074603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1251/2493] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:51:45.082517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6c6fc-26bf-4f47-88ab-872bbf5eb391 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.083144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6c6fc-26bf-4f47-88ab-872bbf5eb391 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.083431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6c6fc-26bf-4f47-88ab-872bbf5eb391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.103164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f9d6c6fc-26bf-4f47-88ab-872bbf5eb391 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:45.104745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f9d6c6fc-26bf-4f47-88ab-872bbf5eb391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:45.119830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1243/2494] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:45.124173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd256fd-3c3a-4dd8-a2f3-b9f484ef5156 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.136387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd256fd-3c3a-4dd8-a2f3-b9f484ef5156 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.141117 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87274d-7527-4963-aecf-0dec0957637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.141889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87274d-7527-4963-aecf-0dec0957637a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.142244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87274d-7527-4963-aecf-0dec0957637a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.142596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87274d-7527-4963-aecf-0dec0957637a admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.143655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87274d-7527-4963-aecf-0dec0957637a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:45.143849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87274d-7527-4963-aecf-0dec0957637a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.145563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87274d-7527-4963-aecf-0dec0957637a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.160492 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1252/2495] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:45.165075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6c6fc-26bf-4f47-88ab-872bbf5eb391 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.174799 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6c6fc-26bf-4f47-88ab-872bbf5eb391 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.180450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfe9c8-81f2-42c4-83a0-7c923905bac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.181306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfe9c8-81f2-42c4-83a0-7c923905bac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.181594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfe9c8-81f2-42c4-83a0-7c923905bac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.181867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfe9c8-81f2-42c4-83a0-7c923905bac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.182582 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfe9c8-81f2-42c4-83a0-7c923905bac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.189478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfe9c8-81f2-42c4-83a0-7c923905bac8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.224257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1244/2496] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:45 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:45.228480 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87274d-7527-4963-aecf-0dec0957637a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.240100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87274d-7527-4963-aecf-0dec0957637a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.320081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfeeeb-c486-4ac0-8a3f-70a6448f15a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.320879 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfeeeb-c486-4ac0-8a3f-70a6448f15a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.321294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfeeeb-c486-4ac0-8a3f-70a6448f15a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.321669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfeeeb-c486-4ac0-8a3f-70a6448f15a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.322611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfeeeb-c486-4ac0-8a3f-70a6448f15a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.324760 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfeeeb-c486-4ac0-8a3f-70a6448f15a1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.353648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1253/2497] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:45 2022] POST /identity/v3/users => generated 302 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:45.357998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfe9c8-81f2-42c4-83a0-7c923905bac8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.369550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfe9c8-81f2-42c4-83a0-7c923905bac8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.375904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e340506-5ff3-479f-97a2-33fade62d300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.376751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e340506-5ff3-479f-97a2-33fade62d300 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.377137 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e340506-5ff3-479f-97a2-33fade62d300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.377539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e340506-5ff3-479f-97a2-33fade62d300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.378435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e340506-5ff3-479f-97a2-33fade62d300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.380221 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e340506-5ff3-479f-97a2-33fade62d300 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.395680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1245/2498] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:51:45.398000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfeeeb-c486-4ac0-8a3f-70a6448f15a1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.410398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfeeeb-c486-4ac0-8a3f-70a6448f15a1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.415179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ac48b-ff4d-488e-b2ad-afd08441e701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.415899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ac48b-ff4d-488e-b2ad-afd08441e701 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.416192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ac48b-ff4d-488e-b2ad-afd08441e701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.416469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ac48b-ff4d-488e-b2ad-afd08441e701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.417221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ac48b-ff4d-488e-b2ad-afd08441e701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.418893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ac48b-ff4d-488e-b2ad-afd08441e701 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.433585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1254/2499] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:45.435439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e340506-5ff3-479f-97a2-33fade62d300 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.444237 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:45.451899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e340506-5ff3-479f-97a2-33fade62d300 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.466623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c2df2-92da-4450-a3b1-dff8c32beeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.467698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c2df2-92da-4450-a3b1-dff8c32beeda admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.467986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c2df2-92da-4450-a3b1-dff8c32beeda admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.468265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c2df2-92da-4450-a3b1-dff8c32beeda admin admin] PATH_INFO: `/v3/projects/94985a8508e64cd5a6c4cefb7b39c089/users/6542b02a459a4bd1b12718d510b67d15/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.484285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c2df2-92da-4450-a3b1-dff8c32beeda admin admin] RBAC: Authorizing `identity:create_grant(project_id=94985a8508e64cd5a6c4cefb7b39c089, user_id=6542b02a459a4bd1b12718d510b67d15, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.486943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c2df2-92da-4450-a3b1-dff8c32beeda admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.501573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1246/2500] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:45 2022] PUT /identity/v3/projects/94985a8508e64cd5a6c4cefb7b39c089/users/6542b02a459a4bd1b12718d510b67d15/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:45.508288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3dd6-68c9-435e-bd70-593cd9713af7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.508657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3dd6-68c9-435e-bd70-593cd9713af7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.509201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3dd6-68c9-435e-bd70-593cd9713af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.530481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b65b3dd6-68c9-435e-bd70-593cd9713af7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:45.531777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b65b3dd6-68c9-435e-bd70-593cd9713af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:45.594836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1255/2501] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:45.606638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c2df2-92da-4450-a3b1-dff8c32beeda admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.608516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-892c2df2-92da-4450-a3b1-dff8c32beeda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:45.688080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c2df2-92da-4450-a3b1-dff8c32beeda admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.692399 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf9fd0-c0e3-4768-a610-2201e26ebed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.693140 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf9fd0-c0e3-4768-a610-2201e26ebed0 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.693493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf9fd0-c0e3-4768-a610-2201e26ebed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.693879 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf9fd0-c0e3-4768-a610-2201e26ebed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.704143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf9fd0-c0e3-4768-a610-2201e26ebed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.705763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf9fd0-c0e3-4768-a610-2201e26ebed0 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.716808 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1247/2502] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:46.283226 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b3dd6-68c9-435e-bd70-593cd9713af7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.286907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b65b3dd6-68c9-435e-bd70-593cd9713af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:46.306029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b3dd6-68c9-435e-bd70-593cd9713af7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.311000 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.312294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.312582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.312885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] PATH_INFO: `/v3/users/0d648fae81514f87b2c0dd7f494aee46` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.314072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d648fae81514f87b2c0dd7f494aee46)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:46.316179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:46.376453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d648fae81514f87b2c0dd7f494aee46'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.377105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d648fae81514f87b2c0dd7f494aee46'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.382745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:46.397515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d648fae81514f87b2c0dd7f494aee46'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.406391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d648fae81514f87b2c0dd7f494aee46'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.414463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d648fae81514f87b2c0dd7f494aee46'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.421580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1256/2503] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:46 2022] DELETE /identity/v3/users/0d648fae81514f87b2c0dd7f494aee46 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:46.608822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4232b1-296a-4e70-bd97-872b74137696 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.609405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4232b1-296a-4e70-bd97-872b74137696 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.609595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4232b1-296a-4e70-bd97-872b74137696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.641166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cb4232b1-296a-4e70-bd97-872b74137696 None None] MFA Rules not processed for user `6542b02a459a4bd1b12718d510b67d15`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:46.643266 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cb4232b1-296a-4e70-bd97-872b74137696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:46.667046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.669493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:46.758906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1248/2504] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:46.767142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4232b1-296a-4e70-bd97-872b74137696 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.768825 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cb4232b1-296a-4e70-bd97-872b74137696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:46.780736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcb1ff-788a-4d8b-a027-3d4a46c42475 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.786733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.787777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.788143 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.788494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] PATH_INFO: `/v3/projects/c854ecea0b6943b2857d531dfbed04ad` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.789340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4232b1-296a-4e70-bd97-872b74137696 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.790355 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c854ecea0b6943b2857d531dfbed04ad)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:46.794541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:46.795131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da2a04-1be1-4bd1-9cf9-e6a381488d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.796053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2a04-1be1-4bd1-9cf9-e6a381488d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.796555 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2a04-1be1-4bd1-9cf9-e6a381488d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.797040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2a04-1be1-4bd1-9cf9-e6a381488d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.797898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da2a04-1be1-4bd1-9cf9-e6a381488d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:46.800137 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da2a04-1be1-4bd1-9cf9-e6a381488d5f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:46.839553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1249/2505] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:46 2022] POST /identity/v3/projects => generated 403 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:46.840729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da2a04-1be1-4bd1-9cf9-e6a381488d5f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.861574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c854ecea0b6943b2857d531dfbed04ad', 'request_id': 'req-564f7e96-81b6-4bfe-abb5-80ee70040232'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.874909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c854ecea0b6943b2857d531dfbed04ad', 'request_id': 'req-564f7e96-81b6-4bfe-abb5-80ee70040232'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.882503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c854ecea0b6943b2857d531dfbed04ad', 'request_id': 'req-564f7e96-81b6-4bfe-abb5-80ee70040232'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.900125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] The token cache is being invalidate because project c854ecea0b6943b2857d531dfbed04ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:46.900683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.901270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:46.903498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1257/2506] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:46 2022] DELETE /identity/v3/projects/c854ecea0b6943b2857d531dfbed04ad => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:46.907094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.908059 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:46.910598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:46.922809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da2a04-1be1-4bd1-9cf9-e6a381488d5f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.991667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a49557-6f81-4d72-9853-b58a338120ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.992358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a49557-6f81-4d72-9853-b58a338120ba admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.992938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a49557-6f81-4d72-9853-b58a338120ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.993065 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a49557-6f81-4d72-9853-b58a338120ba admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.993762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a49557-6f81-4d72-9853-b58a338120ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:46.994944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f7e96-81b6-4bfe-abb5-80ee70040232 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.995760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a49557-6f81-4d72-9853-b58a338120ba admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.000220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.001213 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.001502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.001781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] PATH_INFO: `/v3/users/c849716fb81d438d854cd2805b1aef71` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.003099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c849716fb81d438d854cd2805b1aef71)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.005087 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.006756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:47.018995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1250/2507] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:46 2022] POST /identity/v3/users => generated 308 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:47.019960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a49557-6f81-4d72-9853-b58a338120ba admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.021573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f7a49557-6f81-4d72-9853-b58a338120ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.033512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a49557-6f81-4d72-9853-b58a338120ba admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.039595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.040471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.040757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.041040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] PATH_INFO: `/v3/users/577e30bf83d2471392b170b45d8da0a9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.042691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=577e30bf83d2471392b170b45d8da0a9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.045201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.060364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c849716fb81d438d854cd2805b1aef71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.065924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c849716fb81d438d854cd2805b1aef71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.081051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c849716fb81d438d854cd2805b1aef71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.088541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c849716fb81d438d854cd2805b1aef71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.096005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '577e30bf83d2471392b170b45d8da0a9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.096280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c849716fb81d438d854cd2805b1aef71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.096759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '577e30bf83d2471392b170b45d8da0a9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.102228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1258/2508] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:46 2022] DELETE /identity/v3/users/c849716fb81d438d854cd2805b1aef71 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:47.103006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.104442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.110878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '577e30bf83d2471392b170b45d8da0a9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.119549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '577e30bf83d2471392b170b45d8da0a9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.120403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c845f9f-7e59-4839-9fec-db95532a68e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '577e30bf83d2471392b170b45d8da0a9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.137057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1251/2509] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:47 2022] DELETE /identity/v3/users/577e30bf83d2471392b170b45d8da0a9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:47.141681 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410970f-93b3-45e1-9d26-8ef29d8afb78 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.141681 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410970f-93b3-45e1-9d26-8ef29d8afb78 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.141948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410970f-93b3-45e1-9d26-8ef29d8afb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.161238 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2410970f-93b3-45e1-9d26-8ef29d8afb78 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:47.162452 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2410970f-93b3-45e1-9d26-8ef29d8afb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.180055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214b2e9-5961-455f-b19d-2108a207e316 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.218571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:47.229493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1252/2510] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:51:47.240170 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410970f-93b3-45e1-9d26-8ef29d8afb78 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.241584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2410970f-93b3-45e1-9d26-8ef29d8afb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.250692 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c64317-f5c9-40d0-8ddf-5f96f0de117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.251399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c64317-f5c9-40d0-8ddf-5f96f0de117f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.251722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c64317-f5c9-40d0-8ddf-5f96f0de117f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.252690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c64317-f5c9-40d0-8ddf-5f96f0de117f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.252868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c64317-f5c9-40d0-8ddf-5f96f0de117f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.254605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c64317-f5c9-40d0-8ddf-5f96f0de117f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.267504 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1259/2511] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:47.272430 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c64317-f5c9-40d0-8ddf-5f96f0de117f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.274465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-09c64317-f5c9-40d0-8ddf-5f96f0de117f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.287276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c64317-f5c9-40d0-8ddf-5f96f0de117f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.293609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc93f0e-1e56-4a18-9a6b-f56754318f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.294534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc93f0e-1e56-4a18-9a6b-f56754318f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.294819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc93f0e-1e56-4a18-9a6b-f56754318f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.295488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc93f0e-1e56-4a18-9a6b-f56754318f67 admin admin] PATH_INFO: `/v3/projects/00d7298aef33412b9884ed812f5020f0/users/7abf0d028c0f49e4a5b3e4d31c599be9/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.314264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410970f-93b3-45e1-9d26-8ef29d8afb78 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.315921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc93f0e-1e56-4a18-9a6b-f56754318f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d7298aef33412b9884ed812f5020f0, user_id=7abf0d028c0f49e4a5b3e4d31c599be9, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.318728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc93f0e-1e56-4a18-9a6b-f56754318f67 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.318864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f27e47-759c-4a84-b908-58f167be3114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.321889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f27e47-759c-4a84-b908-58f167be3114 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.321889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f27e47-759c-4a84-b908-58f167be3114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.321889 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f27e47-759c-4a84-b908-58f167be3114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.329681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:47.333042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f27e47-759c-4a84-b908-58f167be3114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.334719 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f27e47-759c-4a84-b908-58f167be3114 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.337700 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1260/2512] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:47 2022] PUT /identity/v3/projects/00d7298aef33412b9884ed812f5020f0/users/7abf0d028c0f49e4a5b3e4d31c599be9/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:47.341025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc93f0e-1e56-4a18-9a6b-f56754318f67 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.343885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4dc93f0e-1e56-4a18-9a6b-f56754318f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.345666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1253/2513] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:51:47.346545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f27e47-759c-4a84-b908-58f167be3114 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.355706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f27e47-759c-4a84-b908-58f167be3114 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.419811 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc93f0e-1e56-4a18-9a6b-f56754318f67 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.427058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0e5c-0be8-4907-a513-1eeddd2007aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.427757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.427966 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0e5c-0be8-4907-a513-1eeddd2007aa admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.428263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0e5c-0be8-4907-a513-1eeddd2007aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.428540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0e5c-0be8-4907-a513-1eeddd2007aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.429405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0e5c-0be8-4907-a513-1eeddd2007aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.430067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.430534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.431037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0e5c-0be8-4907-a513-1eeddd2007aa admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.431228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] PATH_INFO: `/v3/projects/c432d1ff42a048e6838af7b46ff7ec2f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.432781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c432d1ff42a048e6838af7b46ff7ec2f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.437990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.448100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1254/2514] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:47.449212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0e5c-0be8-4907-a513-1eeddd2007aa admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.461050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0e5c-0be8-4907-a513-1eeddd2007aa admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.466418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067ab05-ecf7-4dac-9284-91312c5c9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.467694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067ab05-ecf7-4dac-9284-91312c5c9eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.468277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067ab05-ecf7-4dac-9284-91312c5c9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.468682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067ab05-ecf7-4dac-9284-91312c5c9eab admin admin] PATH_INFO: `/v3/projects/00d7298aef33412b9884ed812f5020f0/users/7abf0d028c0f49e4a5b3e4d31c599be9/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.471723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067ab05-ecf7-4dac-9284-91312c5c9eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d7298aef33412b9884ed812f5020f0, user_id=7abf0d028c0f49e4a5b3e4d31c599be9, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.474507 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067ab05-ecf7-4dac-9284-91312c5c9eab admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.490511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c432d1ff42a048e6838af7b46ff7ec2f', 'request_id': 'req-dcac89dc-39ba-4f99-8835-a4911c7e4870'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.500926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1255/2515] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:47 2022] PUT /identity/v3/projects/00d7298aef33412b9884ed812f5020f0/users/7abf0d028c0f49e4a5b3e4d31c599be9/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:47.502704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c432d1ff42a048e6838af7b46ff7ec2f', 'request_id': 'req-dcac89dc-39ba-4f99-8835-a4911c7e4870'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.509936 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c432d1ff42a048e6838af7b46ff7ec2f', 'request_id': 'req-dcac89dc-39ba-4f99-8835-a4911c7e4870'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.522153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] The token cache is being invalidate because project c432d1ff42a048e6838af7b46ff7ec2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:47.522445 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.523113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dcac89dc-39ba-4f99-8835-a4911c7e4870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.525502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1261/2516] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:47 2022] DELETE /identity/v3/projects/c432d1ff42a048e6838af7b46ff7ec2f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:47.531074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee73a6c-7d6f-4eac-bf61-2264aa400d35 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.531587 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee73a6c-7d6f-4eac-bf61-2264aa400d35 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.532203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee73a6c-7d6f-4eac-bf61-2264aa400d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.555309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6ee73a6c-7d6f-4eac-bf61-2264aa400d35 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:47.559938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6ee73a6c-7d6f-4eac-bf61-2264aa400d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.636370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1256/2517] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:47.638352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d8156-9d20-48cf-9e76-a23ba77b235d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.638570 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d8156-9d20-48cf-9e76-a23ba77b235d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.638820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d8156-9d20-48cf-9e76-a23ba77b235d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.664100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f52d8156-9d20-48cf-9e76-a23ba77b235d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:47.665723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f52d8156-9d20-48cf-9e76-a23ba77b235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.678592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1262/2518] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:51:47.681820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee73a6c-7d6f-4eac-bf61-2264aa400d35 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.692416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee73a6c-7d6f-4eac-bf61-2264aa400d35 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.699043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.699911 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.700194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.700561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.701293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:47.702193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.703892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.727298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1257/2519] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:47.731606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d8156-9d20-48cf-9e76-a23ba77b235d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.743804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d8156-9d20-48cf-9e76-a23ba77b235d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.748964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110fb6b-63b9-4c11-89fc-98dfeb5b06b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.749631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110fb6b-63b9-4c11-89fc-98dfeb5b06b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.749959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110fb6b-63b9-4c11-89fc-98dfeb5b06b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.750234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110fb6b-63b9-4c11-89fc-98dfeb5b06b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.750927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110fb6b-63b9-4c11-89fc-98dfeb5b06b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.752829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.753753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110fb6b-63b9-4c11-89fc-98dfeb5b06b8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.755276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.769122 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ab920-f441-4215-8710-382571b42762 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.775634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.776715 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.777039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.777451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] PATH_INFO: `/v3/projects/52f134e182954c3a8440b2bc6696e2da` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.779825 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f134e182954c3a8440b2bc6696e2da)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.782092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.792280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:47.804377 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1263/2520] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:47 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:47.808031 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110fb6b-63b9-4c11-89fc-98dfeb5b06b8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.844957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f134e182954c3a8440b2bc6696e2da', 'request_id': 'req-b4590d4e-6baa-448c-a187-372814a2bc2d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.851888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f134e182954c3a8440b2bc6696e2da', 'request_id': 'req-b4590d4e-6baa-448c-a187-372814a2bc2d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.857775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f134e182954c3a8440b2bc6696e2da', 'request_id': 'req-b4590d4e-6baa-448c-a187-372814a2bc2d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.879843 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] The token cache is being invalidate because project 52f134e182954c3a8440b2bc6696e2da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:47.880392 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.881214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:47.882945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1258/2521] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:47 2022] DELETE /identity/v3/projects/52f134e182954c3a8440b2bc6696e2da => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:47.887605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.889591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.900717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110fb6b-63b9-4c11-89fc-98dfeb5b06b8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.905494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:47.984584 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca846c7-a86c-4aec-916b-1d0e8aa24e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.985060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4590d4e-6baa-448c-a187-372814a2bc2d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.985884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca846c7-a86c-4aec-916b-1d0e8aa24e04 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.985884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca846c7-a86c-4aec-916b-1d0e8aa24e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.986201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca846c7-a86c-4aec-916b-1d0e8aa24e04 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.987137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca846c7-a86c-4aec-916b-1d0e8aa24e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.989246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca846c7-a86c-4aec-916b-1d0e8aa24e04 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.990929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.991930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.992287 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.992644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] PATH_INFO: `/v3/users/ab894f59a56646af9495c008afd9001f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.994072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab894f59a56646af9495c008afd9001f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.996475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.021640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1264/2522] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:47 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:48.026991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca846c7-a86c-4aec-916b-1d0e8aa24e04 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.029090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-aca846c7-a86c-4aec-916b-1d0e8aa24e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:48.044872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca846c7-a86c-4aec-916b-1d0e8aa24e04 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.051136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f082e3-07f6-46b9-b403-f77d460798f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.051963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f082e3-07f6-46b9-b403-f77d460798f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.052243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f082e3-07f6-46b9-b403-f77d460798f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.052526 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f082e3-07f6-46b9-b403-f77d460798f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.053391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f082e3-07f6-46b9-b403-f77d460798f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.055743 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f082e3-07f6-46b9-b403-f77d460798f8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.070887 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab894f59a56646af9495c008afd9001f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.071614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab894f59a56646af9495c008afd9001f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.076487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1265/2523] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:48.077788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f082e3-07f6-46b9-b403-f77d460798f8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.079636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-22f082e3-07f6-46b9-b403-f77d460798f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:48.087367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab894f59a56646af9495c008afd9001f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.094104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f082e3-07f6-46b9-b403-f77d460798f8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.094806 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab894f59a56646af9495c008afd9001f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.095665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab894f59a56646af9495c008afd9001f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.099300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50868ce-cba4-4ff4-88b3-46e12ca37f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.100058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50868ce-cba4-4ff4-88b3-46e12ca37f6b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.100624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50868ce-cba4-4ff4-88b3-46e12ca37f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.100752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50868ce-cba4-4ff4-88b3-46e12ca37f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.101868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50868ce-cba4-4ff4-88b3-46e12ca37f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.103938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50868ce-cba4-4ff4-88b3-46e12ca37f6b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.112759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1259/2524] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:47 2022] DELETE /identity/v3/users/ab894f59a56646af9495c008afd9001f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:48.116075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1266/2525] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:48.120201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.139224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:48.195426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25371-43ba-4432-874f-358a11af8f79 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.201199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e8f6f-a883-4f2c-b5eb-dec784625258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.204528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e8f6f-a883-4f2c-b5eb-dec784625258 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.204528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e8f6f-a883-4f2c-b5eb-dec784625258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.204837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e8f6f-a883-4f2c-b5eb-dec784625258 admin admin] PATH_INFO: `/v3/projects/b5387bd4427e4a4994d9847c54fbf924/users/4563cbf8fdbc492fae93bf92a004d0bc/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.223293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e8f6f-a883-4f2c-b5eb-dec784625258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5387bd4427e4a4994d9847c54fbf924, user_id=4563cbf8fdbc492fae93bf92a004d0bc, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.226524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e8f6f-a883-4f2c-b5eb-dec784625258 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.242847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1260/2526] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:48 2022] PUT /identity/v3/projects/b5387bd4427e4a4994d9847c54fbf924/users/4563cbf8fdbc492fae93bf92a004d0bc/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:48.248023 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9db4e7-0d01-49ac-add4-717c3271e7cb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.248499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9db4e7-0d01-49ac-add4-717c3271e7cb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.248922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9db4e7-0d01-49ac-add4-717c3271e7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.271635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4f9db4e7-0d01-49ac-add4-717c3271e7cb None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:48.273380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4f9db4e7-0d01-49ac-add4-717c3271e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:48.316543 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9565e-e3b0-4981-8297-658b9429c4bf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.316834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9565e-e3b0-4981-8297-658b9429c4bf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.317114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9565e-e3b0-4981-8297-658b9429c4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.346313 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-42f9565e-e3b0-4981-8297-658b9429c4bf None None] MFA Rules not processed for user `7abf0d028c0f49e4a5b3e4d31c599be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:48.348341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-42f9565e-e3b0-4981-8297-658b9429c4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:48.366257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1267/2527] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:51:48.377585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9db4e7-0d01-49ac-add4-717c3271e7cb None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.379074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4f9db4e7-0d01-49ac-add4-717c3271e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:48.463217 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9db4e7-0d01-49ac-add4-717c3271e7cb None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.467793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b914066-170a-4bac-92d1-82ac98dc7015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.468593 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b914066-170a-4bac-92d1-82ac98dc7015 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.469001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b914066-170a-4bac-92d1-82ac98dc7015 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.469378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b914066-170a-4bac-92d1-82ac98dc7015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.471617 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1261/2528] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:51:48.477007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9565e-e3b0-4981-8297-658b9429c4bf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.478595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-42f9565e-e3b0-4981-8297-658b9429c4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:48.495622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9565e-e3b0-4981-8297-658b9429c4bf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.501805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.502768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.503080 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.503358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] PATH_INFO: `/v3/projects/b9a6ba4fded14525ac507be850bcec3b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.512435 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a6ba4fded14525ac507be850bcec3b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.514700 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.529488 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b914066-170a-4bac-92d1-82ac98dc7015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.531782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b914066-170a-4bac-92d1-82ac98dc7015 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.544781 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1268/2529] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:51:48.572273 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a6ba4fded14525ac507be850bcec3b', 'request_id': 'req-eaed2842-278c-4068-9b1a-adf763c590db'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.580565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a6ba4fded14525ac507be850bcec3b', 'request_id': 'req-eaed2842-278c-4068-9b1a-adf763c590db'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.587777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a6ba4fded14525ac507be850bcec3b', 'request_id': 'req-eaed2842-278c-4068-9b1a-adf763c590db'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.603200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] The token cache is being invalidate because project b9a6ba4fded14525ac507be850bcec3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:48.603528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.604186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.605789 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1262/2530] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:48 2022] DELETE /identity/v3/projects/b9a6ba4fded14525ac507be850bcec3b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:48.610577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b914066-170a-4bac-92d1-82ac98dc7015 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.623661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b914066-170a-4bac-92d1-82ac98dc7015 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.643424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:48.710228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.711565 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.711863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.712147 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] PATH_INFO: `/v3/users/e0f039a3eb6845f297c1b693a052dfb8` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.713423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f039a3eb6845f297c1b693a052dfb8)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.715614 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.793074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f039a3eb6845f297c1b693a052dfb8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.793744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f039a3eb6845f297c1b693a052dfb8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.813112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f039a3eb6845f297c1b693a052dfb8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.821262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f039a3eb6845f297c1b693a052dfb8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.832006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f039a3eb6845f297c1b693a052dfb8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:48.839334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1269/2531] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:48 2022] DELETE /identity/v3/users/e0f039a3eb6845f297c1b693a052dfb8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:49.402027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.403951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:49.420010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed2842-278c-4068-9b1a-adf763c590db admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.428650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:49.514983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.515896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.516641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.516980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] PATH_INFO: `/v3/projects/de7779237e414aa1814f9169736087cc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.519441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7779237e414aa1814f9169736087cc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.522260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.607608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7779237e414aa1814f9169736087cc', 'request_id': 'req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.614872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7779237e414aa1814f9169736087cc', 'request_id': 'req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.623853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7779237e414aa1814f9169736087cc', 'request_id': 'req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.647190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.648771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] The token cache is being invalidate because project de7779237e414aa1814f9169736087cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:49.648903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:49.649077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.649753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ad2bf6b5-3e2b-4c51-bc91-cdcc5f0f59dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.659560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1263/2532] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:49 2022] DELETE /identity/v3/projects/de7779237e414aa1814f9169736087cc => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:49.661464 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6560e-a333-4184-9326-43bf81201727 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.661760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6560e-a333-4184-9326-43bf81201727 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.662048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6560e-a333-4184-9326-43bf81201727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.671048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ceb57d-f29a-4298-b775-0822fc23215a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.704220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-69f6560e-a333-4184-9326-43bf81201727 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:49.705937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-69f6560e-a333-4184-9326-43bf81201727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:49.749810 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.750729 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.751013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.751294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] PATH_INFO: `/v3/users/6542b02a459a4bd1b12718d510b67d15` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.754125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6542b02a459a4bd1b12718d510b67d15)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.756165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.789744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1264/2533] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:49.800024 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a628f1b-6395-46f1-a7c2-d0935e9ad9f6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.800396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a628f1b-6395-46f1-a7c2-d0935e9ad9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.800606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a628f1b-6395-46f1-a7c2-d0935e9ad9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.823320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5a628f1b-6395-46f1-a7c2-d0935e9ad9f6 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:49.824569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:49.824721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5a628f1b-6395-46f1-a7c2-d0935e9ad9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:49.834715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6542b02a459a4bd1b12718d510b67d15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.836039 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6542b02a459a4bd1b12718d510b67d15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.839571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1265/2534] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:49.840525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a628f1b-6395-46f1-a7c2-d0935e9ad9f6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.852788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a628f1b-6395-46f1-a7c2-d0935e9ad9f6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.855198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6542b02a459a4bd1b12718d510b67d15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.859534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.860372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.860672 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.860956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.861774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:49.862722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.865277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6542b02a459a4bd1b12718d510b67d15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.865438 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.877547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6542b02a459a4bd1b12718d510b67d15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:49.886835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1266/2535] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:49.889467 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1270/2536] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:49 2022] DELETE /identity/v3/users/6542b02a459a4bd1b12718d510b67d15 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:49.893780 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.895597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:49.911846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99043abe-951a-4f77-a89b-356bf7b62101 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.931348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:50.007388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad30f4-cc8e-41b9-8918-97585df37b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.008483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad30f4-cc8e-41b9-8918-97585df37b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.008784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad30f4-cc8e-41b9-8918-97585df37b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.009100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad30f4-cc8e-41b9-8918-97585df37b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.009829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad30f4-cc8e-41b9-8918-97585df37b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.011986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad30f4-cc8e-41b9-8918-97585df37b6b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.059085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1267/2537] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:49 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:50.060422 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.078723 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0654b25-8f8b-4016-b9d0-6f799dc70be3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.154766 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52b8f6-191f-49b6-b7ce-30e9cae0f093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.155330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad30f4-cc8e-41b9-8918-97585df37b6b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.155986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b8f6-191f-49b6-b7ce-30e9cae0f093 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.157088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b8f6-191f-49b6-b7ce-30e9cae0f093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.157603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b8f6-191f-49b6-b7ce-30e9cae0f093 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.157782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-38ad30f4-cc8e-41b9-8918-97585df37b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:50.158657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52b8f6-191f-49b6-b7ce-30e9cae0f093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.161679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52b8f6-191f-49b6-b7ce-30e9cae0f093 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.176050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad30f4-cc8e-41b9-8918-97585df37b6b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.184618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.185629 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.186530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.186822 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] PATH_INFO: `/v3/projects/94985a8508e64cd5a6c4cefb7b39c089` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.193601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94985a8508e64cd5a6c4cefb7b39c089)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.196564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.204905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1271/2538] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:50 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:50.206287 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52b8f6-191f-49b6-b7ce-30e9cae0f093 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.218002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52b8f6-191f-49b6-b7ce-30e9cae0f093 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.223292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04986de5-48f7-4333-a7aa-5ce90cee5ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.224159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04986de5-48f7-4333-a7aa-5ce90cee5ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.224589 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04986de5-48f7-4333-a7aa-5ce90cee5ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.225111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04986de5-48f7-4333-a7aa-5ce90cee5ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.226045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04986de5-48f7-4333-a7aa-5ce90cee5ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.228227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04986de5-48f7-4333-a7aa-5ce90cee5ce2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.240567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1272/2539] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:50.245384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04986de5-48f7-4333-a7aa-5ce90cee5ce2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.258864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04986de5-48f7-4333-a7aa-5ce90cee5ce2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.269962 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94985a8508e64cd5a6c4cefb7b39c089', 'request_id': 'req-a21ab614-c2e3-458b-9680-00b50bf34cd3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.278728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94985a8508e64cd5a6c4cefb7b39c089', 'request_id': 'req-a21ab614-c2e3-458b-9680-00b50bf34cd3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.294027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94985a8508e64cd5a6c4cefb7b39c089', 'request_id': 'req-a21ab614-c2e3-458b-9680-00b50bf34cd3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.319445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] The token cache is being invalidate because project 94985a8508e64cd5a6c4cefb7b39c089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:50.319861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.320757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.322835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1268/2540] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:50 2022] DELETE /identity/v3/projects/94985a8508e64cd5a6c4cefb7b39c089 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:50.327526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.329136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:50.347699 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34278a-cf24-4b14-9b88-3a014c68deae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.348690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34278a-cf24-4b14-9b88-3a014c68deae admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.348690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34278a-cf24-4b14-9b88-3a014c68deae admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.350274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34278a-cf24-4b14-9b88-3a014c68deae admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.352038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34278a-cf24-4b14-9b88-3a014c68deae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.354767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34278a-cf24-4b14-9b88-3a014c68deae admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.382052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1273/2541] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:50.384732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34278a-cf24-4b14-9b88-3a014c68deae admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.387255 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3f34278a-cf24-4b14-9b88-3a014c68deae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:50.406414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34278a-cf24-4b14-9b88-3a014c68deae admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.411954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203b735-444c-4f45-9df8-c6ce0284cdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.412893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203b735-444c-4f45-9df8-c6ce0284cdf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.413178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203b735-444c-4f45-9df8-c6ce0284cdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.413456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203b735-444c-4f45-9df8-c6ce0284cdf7 admin admin] PATH_INFO: `/v3/projects/8e0658148a144559b45e5b8ae62f7432/users/e49ccf2f5a2845cb9cd3d9bcadac81ce/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.419251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ab614-c2e3-458b-9680-00b50bf34cd3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.425228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.426409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.426706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.427341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] PATH_INFO: `/v3/users/7abf0d028c0f49e4a5b3e4d31c599be9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.429029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7abf0d028c0f49e4a5b3e4d31c599be9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.430373 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203b735-444c-4f45-9df8-c6ce0284cdf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0658148a144559b45e5b8ae62f7432, user_id=e49ccf2f5a2845cb9cd3d9bcadac81ce, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.431037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.433108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203b735-444c-4f45-9df8-c6ce0284cdf7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.459506 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1274/2542] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:50 2022] PUT /identity/v3/projects/8e0658148a144559b45e5b8ae62f7432/users/e49ccf2f5a2845cb9cd3d9bcadac81ce/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:50.471552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d64826-540f-4fb0-970f-4caead306f6b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.472553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d64826-540f-4fb0-970f-4caead306f6b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.473064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d64826-540f-4fb0-970f-4caead306f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.477126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:50.492276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:50.542401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a1d64826-540f-4fb0-970f-4caead306f6b None None] MFA Rules not processed for user `e49ccf2f5a2845cb9cd3d9bcadac81ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:50.544182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a1d64826-540f-4fb0-970f-4caead306f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:50.546738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7abf0d028c0f49e4a5b3e4d31c599be9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.547574 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7abf0d028c0f49e4a5b3e4d31c599be9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.574397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7abf0d028c0f49e4a5b3e4d31c599be9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.583601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7abf0d028c0f49e4a5b3e4d31c599be9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.584381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7abf0d028c0f49e4a5b3e4d31c599be9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:50.603553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1269/2543] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:50 2022] DELETE /identity/v3/users/7abf0d028c0f49e4a5b3e4d31c599be9 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:50.639813 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1275/2544] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:50.644429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.646510 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:50.746073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c3b13-5dca-44c1-b860-016602d4bc48 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.751440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fa5e7-f570-4326-a531-4d1c07827707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.752325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fa5e7-f570-4326-a531-4d1c07827707 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.752679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fa5e7-f570-4326-a531-4d1c07827707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.753095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fa5e7-f570-4326-a531-4d1c07827707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.753841 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4fa5e7-f570-4326-a531-4d1c07827707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.755965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4fa5e7-f570-4326-a531-4d1c07827707 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.785717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1270/2545] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:50 2022] POST /identity/v3/projects => generated 389 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:50.789949 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d64826-540f-4fb0-970f-4caead306f6b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.824667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fa5e7-f570-4326-a531-4d1c07827707 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.826888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fc4fa5e7-f570-4326-a531-4d1c07827707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:50.880917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d64826-540f-4fb0-970f-4caead306f6b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.925523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fa5e7-f570-4326-a531-4d1c07827707 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.931084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.931985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.932276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.932730 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] PATH_INFO: `/v3/projects/00d7298aef33412b9884ed812f5020f0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.934151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d7298aef33412b9884ed812f5020f0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.936605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.952379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412338df-f9be-479a-8d74-322f9841c2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.953088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412338df-f9be-479a-8d74-322f9841c2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.953379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412338df-f9be-479a-8d74-322f9841c2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.953663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412338df-f9be-479a-8d74-322f9841c2a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.954447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412338df-f9be-479a-8d74-322f9841c2a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.956796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412338df-f9be-479a-8d74-322f9841c2a8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.983499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1276/2546] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:50 2022] POST /identity/v3/users => generated 301 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:50.985771 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:50.985944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412338df-f9be-479a-8d74-322f9841c2a8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.998738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412338df-f9be-479a-8d74-322f9841c2a8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.004557 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d7298aef33412b9884ed812f5020f0', 'request_id': 'req-12327f35-cf6a-40fd-a6bf-66875a3d3038'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:51.014391 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d7298aef33412b9884ed812f5020f0', 'request_id': 'req-12327f35-cf6a-40fd-a6bf-66875a3d3038'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:51.021726 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d7298aef33412b9884ed812f5020f0', 'request_id': 'req-12327f35-cf6a-40fd-a6bf-66875a3d3038'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:51.039627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] The token cache is being invalidate because project 00d7298aef33412b9884ed812f5020f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:51.040052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:51.040792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-12327f35-cf6a-40fd-a6bf-66875a3d3038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:51.043073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1271/2547] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:50 2022] DELETE /identity/v3/projects/00d7298aef33412b9884ed812f5020f0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:51.047363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27998b8-b99d-462c-99ba-c660173f9e3e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.048975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27998b8-b99d-462c-99ba-c660173f9e3e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.049130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:51.049319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27998b8-b99d-462c-99ba-c660173f9e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.084766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b27998b8-b99d-462c-99ba-c660173f9e3e None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:51.086107 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b27998b8-b99d-462c-99ba-c660173f9e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:51.110789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea340-f6d0-48e7-a51f-b99505472821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.111585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea340-f6d0-48e7-a51f-b99505472821 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.111870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea340-f6d0-48e7-a51f-b99505472821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.112181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea340-f6d0-48e7-a51f-b99505472821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.112921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea340-f6d0-48e7-a51f-b99505472821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.114634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea340-f6d0-48e7-a51f-b99505472821 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.136364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1277/2548] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:51:51.137547 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea340-f6d0-48e7-a51f-b99505472821 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.139300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e5fea340-f6d0-48e7-a51f-b99505472821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:51.159753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea340-f6d0-48e7-a51f-b99505472821 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.165573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65351131-60ba-4c07-a9da-f8eb3ced946e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.166552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65351131-60ba-4c07-a9da-f8eb3ced946e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.166835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65351131-60ba-4c07-a9da-f8eb3ced946e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.167111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65351131-60ba-4c07-a9da-f8eb3ced946e admin admin] PATH_INFO: `/v3/projects/9924c062982c4dea9b3ab17299128752/users/f611017e409043e0a497cd30f2a04050/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.180151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1272/2549] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:51.181978 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795d4b-e49d-4052-b95a-c8d22f7c1dfa None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.183818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795d4b-e49d-4052-b95a-c8d22f7c1dfa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.184429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795d4b-e49d-4052-b95a-c8d22f7c1dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.195353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65351131-60ba-4c07-a9da-f8eb3ced946e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9924c062982c4dea9b3ab17299128752, user_id=f611017e409043e0a497cd30f2a04050, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.200003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65351131-60ba-4c07-a9da-f8eb3ced946e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.205533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5b795d4b-e49d-4052-b95a-c8d22f7c1dfa None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:51.207155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5b795d4b-e49d-4052-b95a-c8d22f7c1dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:51.221989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1273/2550] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:51.225904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1278/2551] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:51 2022] PUT /identity/v3/projects/9924c062982c4dea9b3ab17299128752/users/f611017e409043e0a497cd30f2a04050/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:51.226046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b795d4b-e49d-4052-b95a-c8d22f7c1dfa None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.229767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65351131-60ba-4c07-a9da-f8eb3ced946e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.239380 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b795d4b-e49d-4052-b95a-c8d22f7c1dfa None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.321655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65351131-60ba-4c07-a9da-f8eb3ced946e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.328267 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b237cf7-bc3e-4d8c-88e7-493bd2b87912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.329636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b237cf7-bc3e-4d8c-88e7-493bd2b87912 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.330462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b237cf7-bc3e-4d8c-88e7-493bd2b87912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.331053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b237cf7-bc3e-4d8c-88e7-493bd2b87912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.333888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a068e0-ea49-4d1f-8aa1-14705f019c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.334359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b237cf7-bc3e-4d8c-88e7-493bd2b87912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.334572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a068e0-ea49-4d1f-8aa1-14705f019c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.334858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a068e0-ea49-4d1f-8aa1-14705f019c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.335146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a068e0-ea49-4d1f-8aa1-14705f019c95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.335902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a068e0-ea49-4d1f-8aa1-14705f019c95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:51.336208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a068e0-ea49-4d1f-8aa1-14705f019c95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.336308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b237cf7-bc3e-4d8c-88e7-493bd2b87912 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.337831 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a068e0-ea49-4d1f-8aa1-14705f019c95 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.348404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1279/2552] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:51.352397 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b237cf7-bc3e-4d8c-88e7-493bd2b87912 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.361381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1274/2553] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:51.370110 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a068e0-ea49-4d1f-8aa1-14705f019c95 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.370412 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b237cf7-bc3e-4d8c-88e7-493bd2b87912 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.375762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d2dac-808e-4142-b0f3-4decca285b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.376682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d2dac-808e-4142-b0f3-4decca285b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.378335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d2dac-808e-4142-b0f3-4decca285b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.378637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d2dac-808e-4142-b0f3-4decca285b7f admin admin] PATH_INFO: `/v3/projects/9924c062982c4dea9b3ab17299128752/users/f611017e409043e0a497cd30f2a04050/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.380852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d2dac-808e-4142-b0f3-4decca285b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9924c062982c4dea9b3ab17299128752, user_id=f611017e409043e0a497cd30f2a04050, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.383591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d2dac-808e-4142-b0f3-4decca285b7f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.385430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a068e0-ea49-4d1f-8aa1-14705f019c95 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.395496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1280/2554] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:51 2022] PUT /identity/v3/projects/9924c062982c4dea9b3ab17299128752/users/f611017e409043e0a497cd30f2a04050/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:51.405846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc33c34-6748-435f-9f14-b74807367dc8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.406137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc33c34-6748-435f-9f14-b74807367dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.406416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc33c34-6748-435f-9f14-b74807367dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.407996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be3293-3faf-4371-b11f-89451df226bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.408676 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be3293-3faf-4371-b11f-89451df226bf admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.409007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be3293-3faf-4371-b11f-89451df226bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.409279 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be3293-3faf-4371-b11f-89451df226bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.409979 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be3293-3faf-4371-b11f-89451df226bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.412203 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be3293-3faf-4371-b11f-89451df226bf admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.441562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-dcc33c34-6748-435f-9f14-b74807367dc8 None None] MFA Rules not processed for user `f611017e409043e0a497cd30f2a04050`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:51.443906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dcc33c34-6748-435f-9f14-b74807367dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:51.463980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1275/2555] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:51 2022] POST /identity/v3/projects => generated 355 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:51.469174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be3293-3faf-4371-b11f-89451df226bf admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.523845 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:51.572691 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1281/2556] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:51:51.583072 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be3293-3faf-4371-b11f-89451df226bf admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.584044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc33c34-6748-435f-9f14-b74807367dc8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.585494 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dcc33c34-6748-435f-9f14-b74807367dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:51.590956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8d84f-a2e2-44cd-b226-9d3b75c10bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.591928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8d84f-a2e2-44cd-b226-9d3b75c10bed admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.592384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8d84f-a2e2-44cd-b226-9d3b75c10bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.593366 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8d84f-a2e2-44cd-b226-9d3b75c10bed admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.594068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8d84f-a2e2-44cd-b226-9d3b75c10bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.596744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8d84f-a2e2-44cd-b226-9d3b75c10bed admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.604055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:51.629522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1276/2557] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:51 2022] POST /identity/v3/users => generated 278 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:51.633491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8d84f-a2e2-44cd-b226-9d3b75c10bed admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.646433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8d84f-a2e2-44cd-b226-9d3b75c10bed admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.669833 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4d1d2-9690-4769-ba34-665057a25f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.674054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4d1d2-9690-4769-ba34-665057a25f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.674405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4d1d2-9690-4769-ba34-665057a25f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.674753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4d1d2-9690-4769-ba34-665057a25f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.675706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4d1d2-9690-4769-ba34-665057a25f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.680218 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4d1d2-9690-4769-ba34-665057a25f48 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.684514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc33c34-6748-435f-9f14-b74807367dc8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.715126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1277/2558] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:51.719323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4d1d2-9690-4769-ba34-665057a25f48 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.739208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4d1d2-9690-4769-ba34-665057a25f48 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.749580 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0041f-7a48-4257-9a55-ef9a60895c76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.750435 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0041f-7a48-4257-9a55-ef9a60895c76 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.750945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0041f-7a48-4257-9a55-ef9a60895c76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.751320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0041f-7a48-4257-9a55-ef9a60895c76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.755421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2fcbc-cbcf-4f7f-901c-351682e959b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.756139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2fcbc-cbcf-4f7f-901c-351682e959b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.756428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2fcbc-cbcf-4f7f-901c-351682e959b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.756713 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2fcbc-cbcf-4f7f-901c-351682e959b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.757461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2fcbc-cbcf-4f7f-901c-351682e959b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.759095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2fcbc-cbcf-4f7f-901c-351682e959b2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.772514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1278/2559] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:51.776544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2fcbc-cbcf-4f7f-901c-351682e959b2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.789599 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2fcbc-cbcf-4f7f-901c-351682e959b2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.795184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f13ae-ebfb-4ec8-a575-e3ae9ef555a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.796409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f13ae-ebfb-4ec8-a575-e3ae9ef555a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.796727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f13ae-ebfb-4ec8-a575-e3ae9ef555a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.796996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f13ae-ebfb-4ec8-a575-e3ae9ef555a2 admin admin] PATH_INFO: `/v3/projects/347f36da5abd415db96605e2f486dc95/users/9ac909faacd848a79b5eae0bb9d47ba3/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.819258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f13ae-ebfb-4ec8-a575-e3ae9ef555a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347f36da5abd415db96605e2f486dc95, user_id=9ac909faacd848a79b5eae0bb9d47ba3, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.825263 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f13ae-ebfb-4ec8-a575-e3ae9ef555a2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.851932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1279/2560] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:51 2022] PUT /identity/v3/projects/347f36da5abd415db96605e2f486dc95/users/9ac909faacd848a79b5eae0bb9d47ba3/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:51.860076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18601460-d9dd-4da0-8127-953ecd556809 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.861246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18601460-d9dd-4da0-8127-953ecd556809 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.861864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18601460-d9dd-4da0-8127-953ecd556809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.862021 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0041f-7a48-4257-9a55-ef9a60895c76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.864172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0041f-7a48-4257-9a55-ef9a60895c76 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.875310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1282/2561] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:51:51.901202 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-18601460-d9dd-4da0-8127-953ecd556809 None None] MFA Rules not processed for user `9ac909faacd848a79b5eae0bb9d47ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:51.902651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-18601460-d9dd-4da0-8127-953ecd556809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:51.999533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1280/2562] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:51 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:52.002347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0041f-7a48-4257-9a55-ef9a60895c76 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.014806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0041f-7a48-4257-9a55-ef9a60895c76 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.035452 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18601460-d9dd-4da0-8127-953ecd556809 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.136956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18601460-d9dd-4da0-8127-953ecd556809 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.144776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.145109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dbded-0c92-42fd-87bd-cc30d85f4613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.146042 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dbded-0c92-42fd-87bd-cc30d85f4613 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.146161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.146590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dbded-0c92-42fd-87bd-cc30d85f4613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.146720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.147100 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] PATH_INFO: `/v3/users/e49ccf2f5a2845cb9cd3d9bcadac81ce` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.147288 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dbded-0c92-42fd-87bd-cc30d85f4613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.148224 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dbded-0c92-42fd-87bd-cc30d85f4613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.149598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49ccf2f5a2845cb9cd3d9bcadac81ce)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.150889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dbded-0c92-42fd-87bd-cc30d85f4613 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.152502 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.155818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.198368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1283/2563] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:52 2022] POST /identity/v3/projects => generated 355 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:52.203925 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dbded-0c92-42fd-87bd-cc30d85f4613 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.212858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.270615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49ccf2f5a2845cb9cd3d9bcadac81ce'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:52.271565 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49ccf2f5a2845cb9cd3d9bcadac81ce'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:52.293930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49ccf2f5a2845cb9cd3d9bcadac81ce'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:52.297903 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dbded-0c92-42fd-87bd-cc30d85f4613 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.304711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49ccf2f5a2845cb9cd3d9bcadac81ce'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:52.305769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b02eba87-67e4-487c-8e34-4138f0b49892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49ccf2f5a2845cb9cd3d9bcadac81ce'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:52.308060 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcdd40-ea68-42b4-87ab-1a02fb8cdad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.309105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcdd40-ea68-42b4-87ab-1a02fb8cdad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.309802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcdd40-ea68-42b4-87ab-1a02fb8cdad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.310712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcdd40-ea68-42b4-87ab-1a02fb8cdad3 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.311824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcdd40-ea68-42b4-87ab-1a02fb8cdad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.316168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcdd40-ea68-42b4-87ab-1a02fb8cdad3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.330290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1281/2564] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:52 2022] DELETE /identity/v3/users/e49ccf2f5a2845cb9cd3d9bcadac81ce => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:52.333216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd1fff-8b65-46ba-900e-9d715fec7601 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.333682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd1fff-8b65-46ba-900e-9d715fec7601 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.334404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd1fff-8b65-46ba-900e-9d715fec7601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.359092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f3bd1fff-8b65-46ba-900e-9d715fec7601 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:52.360333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f3bd1fff-8b65-46ba-900e-9d715fec7601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:52.398510 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1284/2565] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:52 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:52.401026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcdd40-ea68-42b4-87ab-1a02fb8cdad3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.411953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-aedcdd40-ea68-42b4-87ab-1a02fb8cdad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:52.441299 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1282/2566] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:52.442618 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd1fff-8b65-46ba-900e-9d715fec7601 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.444812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f3bd1fff-8b65-46ba-900e-9d715fec7601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:52.458123 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcdd40-ea68-42b4-87ab-1a02fb8cdad3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.473951 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcba66-9155-40f6-9587-84c1c0e30002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.476157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcba66-9155-40f6-9587-84c1c0e30002 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.476899 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcba66-9155-40f6-9587-84c1c0e30002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.477639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcba66-9155-40f6-9587-84c1c0e30002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.481089 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcba66-9155-40f6-9587-84c1c0e30002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.484279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcba66-9155-40f6-9587-84c1c0e30002 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.503539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1285/2567] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:52.508550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcba66-9155-40f6-9587-84c1c0e30002 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.520007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd1fff-8b65-46ba-900e-9d715fec7601 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.525078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5504e4e-9b06-44cf-b997-345cd3319d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.525764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5504e4e-9b06-44cf-b997-345cd3319d81 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.526686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5504e4e-9b06-44cf-b997-345cd3319d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.526993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5504e4e-9b06-44cf-b997-345cd3319d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.539598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5504e4e-9b06-44cf-b997-345cd3319d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.541078 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5504e4e-9b06-44cf-b997-345cd3319d81 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.548316 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcba66-9155-40f6-9587-84c1c0e30002 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.557599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76570f94-2cb5-45bc-b456-d20dc745068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.558575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76570f94-2cb5-45bc-b456-d20dc745068c admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.558863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76570f94-2cb5-45bc-b456-d20dc745068c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.559142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76570f94-2cb5-45bc-b456-d20dc745068c admin admin] PATH_INFO: `/v3/projects/7d3dab7b71ff4e8c86376b88f42fa837/users/b23499c05fbc4dca8c868798b4738946/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.569208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1283/2568] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:52.585396 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76570f94-2cb5-45bc-b456-d20dc745068c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3dab7b71ff4e8c86376b88f42fa837, user_id=b23499c05fbc4dca8c868798b4738946, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.589059 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76570f94-2cb5-45bc-b456-d20dc745068c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.620038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1286/2569] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:52 2022] PUT /identity/v3/projects/7d3dab7b71ff4e8c86376b88f42fa837/users/b23499c05fbc4dca8c868798b4738946/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:52.621084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5504e4e-9b06-44cf-b997-345cd3319d81 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.633995 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5504e4e-9b06-44cf-b997-345cd3319d81 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.664927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.734015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc365f6c-612d-46d1-971d-3d9daf0efd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.734725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc365f6c-612d-46d1-971d-3d9daf0efd89 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.735036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc365f6c-612d-46d1-971d-3d9daf0efd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.735474 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc365f6c-612d-46d1-971d-3d9daf0efd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.736466 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc365f6c-612d-46d1-971d-3d9daf0efd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.738515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc365f6c-612d-46d1-971d-3d9daf0efd89 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.762648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1284/2570] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:51:52.766753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76570f94-2cb5-45bc-b456-d20dc745068c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.794082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76570f94-2cb5-45bc-b456-d20dc745068c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.800189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcdd13-64a4-4a91-ac02-695664fe696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.801364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcdd13-64a4-4a91-ac02-695664fe696c admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.801717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcdd13-64a4-4a91-ac02-695664fe696c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.802083 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcdd13-64a4-4a91-ac02-695664fe696c admin admin] PATH_INFO: `/v3/projects/7d3dab7b71ff4e8c86376b88f42fa837/users/b23499c05fbc4dca8c868798b4738946/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.804397 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcdd13-64a4-4a91-ac02-695664fe696c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3dab7b71ff4e8c86376b88f42fa837, user_id=b23499c05fbc4dca8c868798b4738946, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.807698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcdd13-64a4-4a91-ac02-695664fe696c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.821222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1287/2571] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:52 2022] PUT /identity/v3/projects/7d3dab7b71ff4e8c86376b88f42fa837/users/b23499c05fbc4dca8c868798b4738946/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:52.832316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341dc2-e1ec-4602-aa2b-700de26084ce None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.832836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341dc2-e1ec-4602-aa2b-700de26084ce None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.833752 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341dc2-e1ec-4602-aa2b-700de26084ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.875093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fe341dc2-e1ec-4602-aa2b-700de26084ce None None] MFA Rules not processed for user `b23499c05fbc4dca8c868798b4738946`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:52.877949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fe341dc2-e1ec-4602-aa2b-700de26084ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.004597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcdd13-64a4-4a91-ac02-695664fe696c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.006678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-53dcdd13-64a4-4a91-ac02-695664fe696c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.058084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1285/2572] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:52 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:53.081813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe341dc2-e1ec-4602-aa2b-700de26084ce None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.084655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fe341dc2-e1ec-4602-aa2b-700de26084ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.133111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcdd13-64a4-4a91-ac02-695664fe696c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.141776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.143987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.144816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.144816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] PATH_INFO: `/v3/projects/8e0658148a144559b45e5b8ae62f7432` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.146674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e0658148a144559b45e5b8ae62f7432)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.149410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.219496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.222026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e0658148a144559b45e5b8ae62f7432', 'request_id': 'req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.234148 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe341dc2-e1ec-4602-aa2b-700de26084ce None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.234748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e0658148a144559b45e5b8ae62f7432', 'request_id': 'req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.247135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e0658148a144559b45e5b8ae62f7432', 'request_id': 'req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.266381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] The token cache is being invalidate because project 8e0658148a144559b45e5b8ae62f7432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:53.266788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.267599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.271192 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1288/2573] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:53 2022] DELETE /identity/v3/projects/8e0658148a144559b45e5b8ae62f7432 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:53.276790 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.279546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.310459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.311152 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.311440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.311757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.318036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.340231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.374028 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b43f3a-69aa-4997-8c8e-8a2f056d74cd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.380138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.385745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.386055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.386348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] PATH_INFO: `/v3/users/f611017e409043e0a497cd30f2a04050` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.398826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f611017e409043e0a497cd30f2a04050)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.401651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.436998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.440563 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.454429 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1286/2574] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:53 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:53.468514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f611017e409043e0a497cd30f2a04050'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.469304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f611017e409043e0a497cd30f2a04050'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.486446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f611017e409043e0a497cd30f2a04050'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.495859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f611017e409043e0a497cd30f2a04050'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.526950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f89c5f8-b906-41ca-95ef-d771280610b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f611017e409043e0a497cd30f2a04050'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:53.531946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1289/2575] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:53 2022] DELETE /identity/v3/users/f611017e409043e0a497cd30f2a04050 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:53.578932 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.580694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.600470 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037164e7-c6e7-41d8-bbc9-db192ab9ebbe service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.678896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.680055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.680055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.680055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.681986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.702088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28391a8c-ef28-43c0-81b7-f9c7a51f7638 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.702562 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28391a8c-ef28-43c0-81b7-f9c7a51f7638 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.702930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28391a8c-ef28-43c0-81b7-f9c7a51f7638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.736898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-28391a8c-ef28-43c0-81b7-f9c7a51f7638 None None] MFA Rules not processed for user `4563cbf8fdbc492fae93bf92a004d0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:53.738924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-28391a8c-ef28-43c0-81b7-f9c7a51f7638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.777157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.778615 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.794379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.796275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1287/2576] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:53 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:53.826920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1290/2577] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:53.831519 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.833013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.875044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28391a8c-ef28-43c0-81b7-f9c7a51f7638 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.876966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-28391a8c-ef28-43c0-81b7-f9c7a51f7638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:53.968015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6517e84-beb5-46a0-8891-0598613e868e service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.984257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc862c72-6440-4a74-8e1f-8d3666135641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.985613 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc862c72-6440-4a74-8e1f-8d3666135641 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.986096 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc862c72-6440-4a74-8e1f-8d3666135641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.986563 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc862c72-6440-4a74-8e1f-8d3666135641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.988415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc862c72-6440-4a74-8e1f-8d3666135641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.991703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc862c72-6440-4a74-8e1f-8d3666135641 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.012249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:54.032677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28391a8c-ef28-43c0-81b7-f9c7a51f7638 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.038950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.039389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1288/2578] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:53 2022] POST /identity/v3/projects => generated 379 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:54.040035 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.040421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.040800 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] PATH_INFO: `/v3/projects/9924c062982c4dea9b3ab17299128752` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.042243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9924c062982c4dea9b3ab17299128752)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.043542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc862c72-6440-4a74-8e1f-8d3666135641 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.044477 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.103907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9924c062982c4dea9b3ab17299128752', 'request_id': 'req-a4a4661c-2c34-4589-88a8-c243c6c2988e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:54.113344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9924c062982c4dea9b3ab17299128752', 'request_id': 'req-a4a4661c-2c34-4589-88a8-c243c6c2988e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:54.120247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc862c72-6440-4a74-8e1f-8d3666135641 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.121303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9924c062982c4dea9b3ab17299128752', 'request_id': 'req-a4a4661c-2c34-4589-88a8-c243c6c2988e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:54.136797 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] The token cache is being invalidate because project 9924c062982c4dea9b3ab17299128752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:54.137157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:54.137965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4a4661c-2c34-4589-88a8-c243c6c2988e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:54.140500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1291/2579] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:53 2022] DELETE /identity/v3/projects/9924c062982c4dea9b3ab17299128752 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:54.145265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1eb12-2391-43e3-8d60-d327a91f887b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.145641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1eb12-2391-43e3-8d60-d327a91f887b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.145998 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1eb12-2391-43e3-8d60-d327a91f887b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.167559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fcb1eb12-2391-43e3-8d60-d327a91f887b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:54.169197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fcb1eb12-2391-43e3-8d60-d327a91f887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:54.192714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab631171-42d2-445b-9733-15b961907de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.193233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab631171-42d2-445b-9733-15b961907de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.193517 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab631171-42d2-445b-9733-15b961907de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.193825 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab631171-42d2-445b-9733-15b961907de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.194533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab631171-42d2-445b-9733-15b961907de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.196546 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab631171-42d2-445b-9733-15b961907de0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.221284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1289/2580] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:54 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:54.222943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab631171-42d2-445b-9733-15b961907de0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.224930 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ab631171-42d2-445b-9733-15b961907de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:54.236242 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1292/2581] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:51:54.238777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab631171-42d2-445b-9733-15b961907de0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.244160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0247b-c566-4c47-aff2-8c14f0e6a701 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.244647 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0247b-c566-4c47-aff2-8c14f0e6a701 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.245108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0247b-c566-4c47-aff2-8c14f0e6a701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.247016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e741d-3ce6-41e6-aac5-a83787c11802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.247877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e741d-3ce6-41e6-aac5-a83787c11802 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.248175 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e741d-3ce6-41e6-aac5-a83787c11802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.248511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e741d-3ce6-41e6-aac5-a83787c11802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.249383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e741d-3ce6-41e6-aac5-a83787c11802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.251216 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e741d-3ce6-41e6-aac5-a83787c11802 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.262716 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1290/2582] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:54.267319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e741d-3ce6-41e6-aac5-a83787c11802 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.270531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e4c0247b-c566-4c47-aff2-8c14f0e6a701 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:54.271805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e4c0247b-c566-4c47-aff2-8c14f0e6a701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:54.278796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e741d-3ce6-41e6-aac5-a83787c11802 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.283897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690d14f-b2a0-4de5-b86b-f25c87970790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.284283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1293/2583] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:54.285746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0247b-c566-4c47-aff2-8c14f0e6a701 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.286134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690d14f-b2a0-4de5-b86b-f25c87970790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.286516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690d14f-b2a0-4de5-b86b-f25c87970790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.286891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690d14f-b2a0-4de5-b86b-f25c87970790 admin admin] PATH_INFO: `/v3/projects/a7efefb29e864f08aca232bab601567d/users/f58cd3f098c043848f8a057dc7f2b93c/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.295972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0247b-c566-4c47-aff2-8c14f0e6a701 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.301517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb7755-8a9f-4b1e-88f2-d4693c602f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.302201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb7755-8a9f-4b1e-88f2-d4693c602f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.302496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb7755-8a9f-4b1e-88f2-d4693c602f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.302779 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb7755-8a9f-4b1e-88f2-d4693c602f9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.303569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb7755-8a9f-4b1e-88f2-d4693c602f9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:54.304049 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb7755-8a9f-4b1e-88f2-d4693c602f9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.305528 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690d14f-b2a0-4de5-b86b-f25c87970790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7efefb29e864f08aca232bab601567d, user_id=f58cd3f098c043848f8a057dc7f2b93c, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.305898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb7755-8a9f-4b1e-88f2-d4693c602f9d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.308599 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690d14f-b2a0-4de5-b86b-f25c87970790 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.308834 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:54.327511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1291/2584] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:54 2022] PUT /identity/v3/projects/a7efefb29e864f08aca232bab601567d/users/f58cd3f098c043848f8a057dc7f2b93c/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:54.328642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690d14f-b2a0-4de5-b86b-f25c87970790 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.330786 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1294/2585] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:54.331884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb7755-8a9f-4b1e-88f2-d4693c602f9d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.342642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb7755-8a9f-4b1e-88f2-d4693c602f9d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.389165 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:54.407007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690d14f-b2a0-4de5-b86b-f25c87970790 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.413872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87aa641-1b22-42be-acbc-08e922c5221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.414829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aa641-1b22-42be-acbc-08e922c5221b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.415194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aa641-1b22-42be-acbc-08e922c5221b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.415579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aa641-1b22-42be-acbc-08e922c5221b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.416026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea65836-96d2-46c0-90f7-14d1fbd1a97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.416489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87aa641-1b22-42be-acbc-08e922c5221b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.416890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65836-96d2-46c0-90f7-14d1fbd1a97a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.417246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65836-96d2-46c0-90f7-14d1fbd1a97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.417623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65836-96d2-46c0-90f7-14d1fbd1a97a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.418438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea65836-96d2-46c0-90f7-14d1fbd1a97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.419040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87aa641-1b22-42be-acbc-08e922c5221b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.420551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea65836-96d2-46c0-90f7-14d1fbd1a97a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.440333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1295/2586] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:54.445257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea65836-96d2-46c0-90f7-14d1fbd1a97a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.460447 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea65836-96d2-46c0-90f7-14d1fbd1a97a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.460652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1292/2587] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:54 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:54.462110 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87aa641-1b22-42be-acbc-08e922c5221b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.563149 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87aa641-1b22-42be-acbc-08e922c5221b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.572163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091594c-8941-433e-8419-febe5980c9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.573491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe1774-cd02-49ef-a3f7-ca4be286dfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.573639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091594c-8941-433e-8419-febe5980c9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.574188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe1774-cd02-49ef-a3f7-ca4be286dfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.574476 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe1774-cd02-49ef-a3f7-ca4be286dfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.574755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe1774-cd02-49ef-a3f7-ca4be286dfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.575486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe1774-cd02-49ef-a3f7-ca4be286dfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.576400 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091594c-8941-433e-8419-febe5980c9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.576920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091594c-8941-433e-8419-febe5980c9b1 admin admin] PATH_INFO: `/v3/projects/a7efefb29e864f08aca232bab601567d/users/f58cd3f098c043848f8a057dc7f2b93c/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.578119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe1774-cd02-49ef-a3f7-ca4be286dfc5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.588300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091594c-8941-433e-8419-febe5980c9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7efefb29e864f08aca232bab601567d, user_id=f58cd3f098c043848f8a057dc7f2b93c, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.593250 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091594c-8941-433e-8419-febe5980c9b1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.614784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1296/2588] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:54 2022] PUT /identity/v3/projects/a7efefb29e864f08aca232bab601567d/users/f58cd3f098c043848f8a057dc7f2b93c/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:54.616159 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1293/2589] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:54 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:54.617344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091594c-8941-433e-8419-febe5980c9b1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.704124 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091594c-8941-433e-8419-febe5980c9b1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.709983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa651983-9854-4360-b9db-a81caa47f208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.710864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa651983-9854-4360-b9db-a81caa47f208 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.711268 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa651983-9854-4360-b9db-a81caa47f208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.711758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa651983-9854-4360-b9db-a81caa47f208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.713058 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa651983-9854-4360-b9db-a81caa47f208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.715130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa651983-9854-4360-b9db-a81caa47f208 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.727954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1297/2590] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:51:54.731949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe1774-cd02-49ef-a3f7-ca4be286dfc5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.743569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe1774-cd02-49ef-a3f7-ca4be286dfc5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.751540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6cd43-7f61-49f8-8b40-3cc44c061ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.752246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6cd43-7f61-49f8-8b40-3cc44c061ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.752542 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6cd43-7f61-49f8-8b40-3cc44c061ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.752905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6cd43-7f61-49f8-8b40-3cc44c061ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.753656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6cd43-7f61-49f8-8b40-3cc44c061ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.755501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6cd43-7f61-49f8-8b40-3cc44c061ec0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.769571 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1294/2591] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:54.773815 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa651983-9854-4360-b9db-a81caa47f208 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:54.784698 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa651983-9854-4360-b9db-a81caa47f208 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:54.789983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e83fb-0998-4cdf-812c-9798d31c6f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:54.791313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e83fb-0998-4cdf-812c-9798d31c6f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.791313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e83fb-0998-4cdf-812c-9798d31c6f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.791598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e83fb-0998-4cdf-812c-9798d31c6f08 admin admin] PATH_INFO: `/v3/projects/06ec0176417e43deb20ee6b852f9c2a5/users/e26531015e3b4339920d0710e78ab47f/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.808876 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e83fb-0998-4cdf-812c-9798d31c6f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ec0176417e43deb20ee6b852f9c2a5, user_id=e26531015e3b4339920d0710e78ab47f, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:54.811953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e83fb-0998-4cdf-812c-9798d31c6f08 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:54.828597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:54.835419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1298/2592] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:54 2022] PUT /identity/v3/projects/06ec0176417e43deb20ee6b852f9c2a5/users/e26531015e3b4339920d0710e78ab47f/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:54.846412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c1d7-8d39-46cc-883c-24d4b1332864 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:54.846870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c1d7-8d39-46cc-883c-24d4b1332864 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:54.847208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c1d7-8d39-46cc-883c-24d4b1332864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:54.879553 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-37b2c1d7-8d39-46cc-883c-24d4b1332864 None None] MFA Rules not processed for user `e26531015e3b4339920d0710e78ab47f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:54.880830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-37b2c1d7-8d39-46cc-883c-24d4b1332864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:54.961116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1295/2593] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:54.965656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e83fb-0998-4cdf-812c-9798d31c6f08 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:55.057913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e83fb-0998-4cdf-812c-9798d31c6f08 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:55.069722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7078-59da-4d4a-8f8f-46e4408154fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:55.070942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7078-59da-4d4a-8f8f-46e4408154fe admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:55.071294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7078-59da-4d4a-8f8f-46e4408154fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:55.071981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7078-59da-4d4a-8f8f-46e4408154fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:55.072759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7078-59da-4d4a-8f8f-46e4408154fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:55.074967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7078-59da-4d4a-8f8f-46e4408154fe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:55.119839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1299/2594] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:54 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:55.120664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c1d7-8d39-46cc-883c-24d4b1332864 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:55.136437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c1d7-8d39-46cc-883c-24d4b1332864 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:55.232473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:55.259839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70468795-b52d-4f0c-a4f0-dcb7ecac7025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:55.260854 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70468795-b52d-4f0c-a4f0-dcb7ecac7025 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:55.260980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70468795-b52d-4f0c-a4f0-dcb7ecac7025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:55.261291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70468795-b52d-4f0c-a4f0-dcb7ecac7025 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:55.262084 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70468795-b52d-4f0c-a4f0-dcb7ecac7025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:55.268248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70468795-b52d-4f0c-a4f0-dcb7ecac7025 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:55.294861 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1296/2595] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:55 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:55.299389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7078-59da-4d4a-8f8f-46e4408154fe admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:55.313018 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7078-59da-4d4a-8f8f-46e4408154fe admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:55.318585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b94952-1105-4861-ba71-3aeaf05d1a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:55.319289 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b94952-1105-4861-ba71-3aeaf05d1a4a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:55.319624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b94952-1105-4861-ba71-3aeaf05d1a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:55.319955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b94952-1105-4861-ba71-3aeaf05d1a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:55.320731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b94952-1105-4861-ba71-3aeaf05d1a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:55.322452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b94952-1105-4861-ba71-3aeaf05d1a4a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:55.340561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1300/2596] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:55.341810 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70468795-b52d-4f0c-a4f0-dcb7ecac7025 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:55.358964 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70468795-b52d-4f0c-a4f0-dcb7ecac7025 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:55.364447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e207574-37cd-46ec-a76b-8dba1d49609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:55.365020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e207574-37cd-46ec-a76b-8dba1d49609b admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:55.365312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e207574-37cd-46ec-a76b-8dba1d49609b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:55.365590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e207574-37cd-46ec-a76b-8dba1d49609b admin admin] PATH_INFO: `/v3/projects/e310a95316a347c585b73c2d4b24b13c/users/2137f8a52a2a453abd23580df2819dff/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:55.394417 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e207574-37cd-46ec-a76b-8dba1d49609b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e310a95316a347c585b73c2d4b24b13c, user_id=2137f8a52a2a453abd23580df2819dff, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:55.399196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e207574-37cd-46ec-a76b-8dba1d49609b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:55.417031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1297/2597] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:55 2022] PUT /identity/v3/projects/e310a95316a347c585b73c2d4b24b13c/users/2137f8a52a2a453abd23580df2819dff/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:55.421941 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b94952-1105-4861-ba71-3aeaf05d1a4a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:55.433364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b94952-1105-4861-ba71-3aeaf05d1a4a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:55.500733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e714fa6-a702-4b59-8d0d-b6b7b7f6f210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:55.502575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e714fa6-a702-4b59-8d0d-b6b7b7f6f210 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:55.502575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e714fa6-a702-4b59-8d0d-b6b7b7f6f210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:55.502575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e714fa6-a702-4b59-8d0d-b6b7b7f6f210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:55.503003 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e714fa6-a702-4b59-8d0d-b6b7b7f6f210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:55.504888 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e714fa6-a702-4b59-8d0d-b6b7b7f6f210 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:55.519633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1301/2598] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:55.521104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e207574-37cd-46ec-a76b-8dba1d49609b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:55.532996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e207574-37cd-46ec-a76b-8dba1d49609b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:55.539648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4eb14-5141-4c70-98d8-4d10061ae3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:55.540572 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4eb14-5141-4c70-98d8-4d10061ae3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:55.540865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4eb14-5141-4c70-98d8-4d10061ae3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:55.541145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4eb14-5141-4c70-98d8-4d10061ae3c3 admin admin] PATH_INFO: `/v3/projects/e310a95316a347c585b73c2d4b24b13c/users/2137f8a52a2a453abd23580df2819dff/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:55.543215 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4eb14-5141-4c70-98d8-4d10061ae3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e310a95316a347c585b73c2d4b24b13c, user_id=2137f8a52a2a453abd23580df2819dff, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:55.545939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4eb14-5141-4c70-98d8-4d10061ae3c3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:55.567443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1298/2599] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:55 2022] PUT /identity/v3/projects/e310a95316a347c585b73c2d4b24b13c/users/2137f8a52a2a453abd23580df2819dff/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:55.570679 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88103f-89e9-4327-9648-2906dbe8b97b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:55.571205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88103f-89e9-4327-9648-2906dbe8b97b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:55.571684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88103f-89e9-4327-9648-2906dbe8b97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:55.607014 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ff88103f-89e9-4327-9648-2906dbe8b97b None None] MFA Rules not processed for user `2137f8a52a2a453abd23580df2819dff`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:55.608527 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ff88103f-89e9-4327-9648-2906dbe8b97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:55.684732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:55.700564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1302/2600] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:55.711187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4eb14-5141-4c70-98d8-4d10061ae3c3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:55.713069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-30a4eb14-5141-4c70-98d8-4d10061ae3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:55.776682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4eb14-5141-4c70-98d8-4d10061ae3c3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:55.781199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58ec91-d2fb-4886-a655-8f0faf611d7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:55.781875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58ec91-d2fb-4886-a655-8f0faf611d7b service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:55.782393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58ec91-d2fb-4886-a655-8f0faf611d7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:55.782782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58ec91-d2fb-4886-a655-8f0faf611d7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:55.797761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58ec91-d2fb-4886-a655-8f0faf611d7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:55.799344 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58ec91-d2fb-4886-a655-8f0faf611d7b service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:55.817012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1299/2601] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:51:56.349864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88103f-89e9-4327-9648-2906dbe8b97b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:56.496525 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88103f-89e9-4327-9648-2906dbe8b97b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:56.531353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:56.532670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.532954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.533236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] PATH_INFO: `/v3/users/e26531015e3b4339920d0710e78ab47f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:56.536476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e26531015e3b4339920d0710e78ab47f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:56.538707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:56.611842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e26531015e3b4339920d0710e78ab47f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:56.612866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e26531015e3b4339920d0710e78ab47f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:56.631706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e26531015e3b4339920d0710e78ab47f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:56.642046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e26531015e3b4339920d0710e78ab47f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:56.653449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e26531015e3b4339920d0710e78ab47f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:56.665785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1303/2602] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:56 2022] DELETE /identity/v3/users/e26531015e3b4339920d0710e78ab47f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:56.668296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fca1c0-7a58-4227-b85e-8acbd7fea60c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.668529 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fca1c0-7a58-4227-b85e-8acbd7fea60c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.668816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fca1c0-7a58-4227-b85e-8acbd7fea60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:56.703751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a4fca1c0-7a58-4227-b85e-8acbd7fea60c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:56.705608 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a4fca1c0-7a58-4227-b85e-8acbd7fea60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:56.813925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1300/2603] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:56.826455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:56.836592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:56.963954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959273c-6f6f-4be5-aac0-a0d90192fca1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:56.974746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2a071-9401-487e-82d2-91d6000e19cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:56.975474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2a071-9401-487e-82d2-91d6000e19cb service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.976115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2a071-9401-487e-82d2-91d6000e19cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.976329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2a071-9401-487e-82d2-91d6000e19cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:56.996778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2a071-9401-487e-82d2-91d6000e19cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:56.998308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2a071-9401-487e-82d2-91d6000e19cb service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.020247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1304/2604] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:51:57.498645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fca1c0-7a58-4227-b85e-8acbd7fea60c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.501709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a4fca1c0-7a58-4227-b85e-8acbd7fea60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:57.523234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fca1c0-7a58-4227-b85e-8acbd7fea60c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.529015 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:57.529923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.530645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.530929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] PATH_INFO: `/v3/projects/06ec0176417e43deb20ee6b852f9c2a5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.532185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06ec0176417e43deb20ee6b852f9c2a5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.534186 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.632952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.652641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06ec0176417e43deb20ee6b852f9c2a5', 'request_id': 'req-daade3d8-682b-49c0-aeeb-b05635a54917'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.665478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06ec0176417e43deb20ee6b852f9c2a5', 'request_id': 'req-daade3d8-682b-49c0-aeeb-b05635a54917'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.672842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06ec0176417e43deb20ee6b852f9c2a5', 'request_id': 'req-daade3d8-682b-49c0-aeeb-b05635a54917'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.700645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] The token cache is being invalidate because project 06ec0176417e43deb20ee6b852f9c2a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:57.700959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.702479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.704549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1301/2605] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:57 2022] DELETE /identity/v3/projects/06ec0176417e43deb20ee6b852f9c2a5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:57.708406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2a071-9401-487e-82d2-91d6000e19cb service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.720015 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2a071-9401-487e-82d2-91d6000e19cb service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.749233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.842237 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:57.843241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.843561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.843848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] PATH_INFO: `/v3/users/2137f8a52a2a453abd23580df2819dff` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.845073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2137f8a52a2a453abd23580df2819dff)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.847405 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.926081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2137f8a52a2a453abd23580df2819dff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.926804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2137f8a52a2a453abd23580df2819dff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.960014 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2137f8a52a2a453abd23580df2819dff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.970347 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2137f8a52a2a453abd23580df2819dff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.989037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a60da0e5-c0b2-499b-81f4-b018a7290296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2137f8a52a2a453abd23580df2819dff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:57.999523 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1305/2606] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:51:57 2022] DELETE /identity/v3/users/2137f8a52a2a453abd23580df2819dff => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:58.516114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:58.517650 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:58.544085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daade3d8-682b-49c0-aeeb-b05635a54917 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:58.690136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:58.691301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.691722 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.692074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] PATH_INFO: `/v3/projects/e310a95316a347c585b73c2d4b24b13c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.700005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e310a95316a347c585b73c2d4b24b13c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:58.702671 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:58.802371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e310a95316a347c585b73c2d4b24b13c', 'request_id': 'req-2fa0c04d-a450-4b72-9b1c-a4676f54f376'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:58.821928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e310a95316a347c585b73c2d4b24b13c', 'request_id': 'req-2fa0c04d-a450-4b72-9b1c-a4676f54f376'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:58.828774 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e310a95316a347c585b73c2d4b24b13c', 'request_id': 'req-2fa0c04d-a450-4b72-9b1c-a4676f54f376'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:58.861425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] The token cache is being invalidate because project e310a95316a347c585b73c2d4b24b13c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:51:58.861997 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:58.862944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2fa0c04d-a450-4b72-9b1c-a4676f54f376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:51:58.865386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1302/2607] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:51:58 2022] DELETE /identity/v3/projects/e310a95316a347c585b73c2d4b24b13c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:58.870130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d44d3-9150-4067-9f4a-4462841f6794 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.870632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d44d3-9150-4067-9f4a-4462841f6794 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.870750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d44d3-9150-4067-9f4a-4462841f6794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.903689 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-260d44d3-9150-4067-9f4a-4462841f6794 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:58.905670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-260d44d3-9150-4067-9f4a-4462841f6794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:59.041650 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1306/2608] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:59.041985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996743b-6253-4b3f-a174-313078bafe8f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.042214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996743b-6253-4b3f-a174-313078bafe8f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.042493 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996743b-6253-4b3f-a174-313078bafe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.084866 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a996743b-6253-4b3f-a174-313078bafe8f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:59.087600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a996743b-6253-4b3f-a174-313078bafe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:59.103890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1303/2609] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:59.108502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d44d3-9150-4067-9f4a-4462841f6794 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:59.120462 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d44d3-9150-4067-9f4a-4462841f6794 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:59.136603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceee13b-c9a0-4530-9884-777530b5e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:59.137313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceee13b-c9a0-4530-9884-777530b5e492 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.137603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceee13b-c9a0-4530-9884-777530b5e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.137891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceee13b-c9a0-4530-9884-777530b5e492 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.138657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceee13b-c9a0-4530-9884-777530b5e492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:59.138972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceee13b-c9a0-4530-9884-777530b5e492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:59.140693 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceee13b-c9a0-4530-9884-777530b5e492 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:59.172501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1307/2610] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:59.173733 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996743b-6253-4b3f-a174-313078bafe8f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:59.190867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996743b-6253-4b3f-a174-313078bafe8f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:59.199633 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c81243-d7f7-4d6c-8e03-a92f41b1818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:59.200361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c81243-d7f7-4d6c-8e03-a92f41b1818f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.200655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c81243-d7f7-4d6c-8e03-a92f41b1818f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.200936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c81243-d7f7-4d6c-8e03-a92f41b1818f admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.201658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c81243-d7f7-4d6c-8e03-a92f41b1818f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:59.203802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c81243-d7f7-4d6c-8e03-a92f41b1818f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:59.262413 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceee13b-c9a0-4530-9884-777530b5e492 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:59.263086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1304/2611] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:51:59 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:59.276161 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceee13b-c9a0-4530-9884-777530b5e492 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:59.310852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:59.375643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e83ab-5646-4311-85c6-4bf9513909a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:59.376487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e83ab-5646-4311-85c6-4bf9513909a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.376863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e83ab-5646-4311-85c6-4bf9513909a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.377243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e83ab-5646-4311-85c6-4bf9513909a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.378174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e83ab-5646-4311-85c6-4bf9513909a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:59.380514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e83ab-5646-4311-85c6-4bf9513909a3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:59.409278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1308/2612] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:51:59 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:59.414653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c81243-d7f7-4d6c-8e03-a92f41b1818f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:59.427093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c81243-d7f7-4d6c-8e03-a92f41b1818f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:59.438070 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c774ca6-c8ea-413e-882e-9890fdeac546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:59.438836 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c774ca6-c8ea-413e-882e-9890fdeac546 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.440206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c774ca6-c8ea-413e-882e-9890fdeac546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.440206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c774ca6-c8ea-413e-882e-9890fdeac546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.440416 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c774ca6-c8ea-413e-882e-9890fdeac546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:59.442012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c774ca6-c8ea-413e-882e-9890fdeac546 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:59.453815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1305/2613] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:59.457976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e83ab-5646-4311-85c6-4bf9513909a3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:59.481577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e83ab-5646-4311-85c6-4bf9513909a3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:59.488831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf76d0-ae54-4545-b19e-d72adbdf3e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:59.491753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf76d0-ae54-4545-b19e-d72adbdf3e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.492298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf76d0-ae54-4545-b19e-d72adbdf3e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.493091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf76d0-ae54-4545-b19e-d72adbdf3e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.494314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf76d0-ae54-4545-b19e-d72adbdf3e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:59.496466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf76d0-ae54-4545-b19e-d72adbdf3e82 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:59.514414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1309/2614] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:51:59.519329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c774ca6-c8ea-413e-882e-9890fdeac546 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:59.536251 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c774ca6-c8ea-413e-882e-9890fdeac546 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:59.542526 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1525e-0b71-4384-9c28-6f58b0eb09a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:59.544112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1525e-0b71-4384-9c28-6f58b0eb09a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.544500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1525e-0b71-4384-9c28-6f58b0eb09a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.544897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1525e-0b71-4384-9c28-6f58b0eb09a8 admin admin] PATH_INFO: `/v3/projects/f840d73c94fa4abc9552d50b0f78fd89/users/7cd67f4fdca84f328a7fe02623304f1c/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.567295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:59.573103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1525e-0b71-4384-9c28-6f58b0eb09a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f840d73c94fa4abc9552d50b0f78fd89, user_id=7cd67f4fdca84f328a7fe02623304f1c, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:59.576969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1525e-0b71-4384-9c28-6f58b0eb09a8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:59.599784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1306/2615] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:51:59 2022] PUT /identity/v3/projects/f840d73c94fa4abc9552d50b0f78fd89/users/7cd67f4fdca84f328a7fe02623304f1c/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:59.605370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc79e-8764-4319-a1b7-cd274be76e00 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.605747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc79e-8764-4319-a1b7-cd274be76e00 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.606101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc79e-8764-4319-a1b7-cd274be76e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.625352 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2ddcc79e-8764-4319-a1b7-cd274be76e00 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:59.626690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2ddcc79e-8764-4319-a1b7-cd274be76e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:59.707596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1310/2616] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:51:59.718724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1525e-0b71-4384-9c28-6f58b0eb09a8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:59.722056 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-94f1525e-0b71-4384-9c28-6f58b0eb09a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:59.824352 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1525e-0b71-4384-9c28-6f58b0eb09a8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:59.829480 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942444b9-3e9c-4f84-8cd5-a413b4244fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:59.830500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942444b9-3e9c-4f84-8cd5-a413b4244fcb service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.830996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942444b9-3e9c-4f84-8cd5-a413b4244fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.831430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942444b9-3e9c-4f84-8cd5-a413b4244fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.845436 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942444b9-3e9c-4f84-8cd5-a413b4244fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:59.847697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942444b9-3e9c-4f84-8cd5-a413b4244fcb service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:59.875560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1307/2617] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:00.251986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411ea58-ac19-4dc2-a317-d1b8ec717399 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.252454 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411ea58-ac19-4dc2-a317-d1b8ec717399 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.252574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411ea58-ac19-4dc2-a317-d1b8ec717399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.281737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0411ea58-ac19-4dc2-a317-d1b8ec717399 None None] MFA Rules not processed for user `f58cd3f098c043848f8a057dc7f2b93c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:00.283074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0411ea58-ac19-4dc2-a317-d1b8ec717399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:00.382423 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1311/2618] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:00.404670 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942444b9-3e9c-4f84-8cd5-a413b4244fcb service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.406293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-942444b9-3e9c-4f84-8cd5-a413b4244fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:00.519264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942444b9-3e9c-4f84-8cd5-a413b4244fcb service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:00.524976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:00.525850 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.526271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.526579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.528475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:00.595188 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:00.597337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:00.606983 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1308/2619] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:00.658202 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352ac46-98a3-4c93-855d-d1bbaa1ec142 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.658487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352ac46-98a3-4c93-855d-d1bbaa1ec142 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.658817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352ac46-98a3-4c93-855d-d1bbaa1ec142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.688577 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0352ac46-98a3-4c93-855d-d1bbaa1ec142 None None] MFA Rules not processed for user `7cd67f4fdca84f328a7fe02623304f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:00.689385 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0352ac46-98a3-4c93-855d-d1bbaa1ec142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:00.703576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:00.771311 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1312/2620] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:00.782583 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.784126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:00.822031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:00.859397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869e92e-9933-4d02-a58c-023638a63406 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:00.863225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c14081-0d34-44ca-bdcc-2f8077bf3977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:00.863971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c14081-0d34-44ca-bdcc-2f8077bf3977 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.864443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c14081-0d34-44ca-bdcc-2f8077bf3977 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.864878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c14081-0d34-44ca-bdcc-2f8077bf3977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.876504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c14081-0d34-44ca-bdcc-2f8077bf3977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:00.880142 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c14081-0d34-44ca-bdcc-2f8077bf3977 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:00.896518 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1309/2621] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:02.472047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e2a0e-d439-42d6-a82f-7280f5700837 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.472557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e2a0e-d439-42d6-a82f-7280f5700837 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.472656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e2a0e-d439-42d6-a82f-7280f5700837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.493148 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a35e2a0e-d439-42d6-a82f-7280f5700837 None None] MFA Rules not processed for user `7cd67f4fdca84f328a7fe02623304f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:02.494387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a35e2a0e-d439-42d6-a82f-7280f5700837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:02.501744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1313/2622] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:05.418141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c14081-0d34-44ca-bdcc-2f8077bf3977 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:05.422339 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-51c14081-0d34-44ca-bdcc-2f8077bf3977 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:05.436832 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c14081-0d34-44ca-bdcc-2f8077bf3977 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:05.441928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac283305-fcfd-4f14-bed9-70c55b9b5157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:05.442686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac283305-fcfd-4f14-bed9-70c55b9b5157 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:05.443098 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac283305-fcfd-4f14-bed9-70c55b9b5157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:05.443485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac283305-fcfd-4f14-bed9-70c55b9b5157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:05.445444 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ac283305-fcfd-4f14-bed9-70c55b9b5157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:05.461651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac283305-fcfd-4f14-bed9-70c55b9b5157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:05.463360 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac283305-fcfd-4f14-bed9-70c55b9b5157 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:05.474323 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1310/2623] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:10.310663 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e2a0e-d439-42d6-a82f-7280f5700837 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.312348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a35e2a0e-d439-42d6-a82f-7280f5700837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:10.327296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e2a0e-d439-42d6-a82f-7280f5700837 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.332551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.333540 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.333835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.334140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] PATH_INFO: `/v3/users/9ac909faacd848a79b5eae0bb9d47ba3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.335675 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac909faacd848a79b5eae0bb9d47ba3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.337736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.388887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac909faacd848a79b5eae0bb9d47ba3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:10.389688 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac909faacd848a79b5eae0bb9d47ba3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:10.404108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac909faacd848a79b5eae0bb9d47ba3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:10.410892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac909faacd848a79b5eae0bb9d47ba3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:10.424203 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac909faacd848a79b5eae0bb9d47ba3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:10.431764 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1314/2624] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:52:10 2022] DELETE /identity/v3/users/9ac909faacd848a79b5eae0bb9d47ba3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:10.436512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166bb4cb-2e74-46cb-9af8-5435a4319b72 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.437025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166bb4cb-2e74-46cb-9af8-5435a4319b72 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.437145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166bb4cb-2e74-46cb-9af8-5435a4319b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.455891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-166bb4cb-2e74-46cb-9af8-5435a4319b72 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:10.457146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-166bb4cb-2e74-46cb-9af8-5435a4319b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:10.521522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1311/2625] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:52:10.532518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.534067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:10.595849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31ffb4-1e99-46b3-bd0d-573ce0e79f95 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.600052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03235e34-3882-4d1e-9975-4efe24367f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.600722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03235e34-3882-4d1e-9975-4efe24367f11 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.601013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03235e34-3882-4d1e-9975-4efe24367f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.601296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03235e34-3882-4d1e-9975-4efe24367f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.610864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03235e34-3882-4d1e-9975-4efe24367f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.612404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03235e34-3882-4d1e-9975-4efe24367f11 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.622256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1315/2626] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:10.916261 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166bb4cb-2e74-46cb-9af8-5435a4319b72 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.917955 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-166bb4cb-2e74-46cb-9af8-5435a4319b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:10.930659 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166bb4cb-2e74-46cb-9af8-5435a4319b72 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.938461 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.939925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.940219 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.940609 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] PATH_INFO: `/v3/projects/347f36da5abd415db96605e2f486dc95` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.942240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347f36da5abd415db96605e2f486dc95)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.944720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.993484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347f36da5abd415db96605e2f486dc95', 'request_id': 'req-5b79a284-a04e-4e6a-a575-62d5edf48ca5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.000623 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347f36da5abd415db96605e2f486dc95', 'request_id': 'req-5b79a284-a04e-4e6a-a575-62d5edf48ca5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.007288 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347f36da5abd415db96605e2f486dc95', 'request_id': 'req-5b79a284-a04e-4e6a-a575-62d5edf48ca5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.020940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] The token cache is being invalidate because project 347f36da5abd415db96605e2f486dc95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:52:11.023907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.023907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.023907 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1312/2627] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:52:10 2022] DELETE /identity/v3/projects/347f36da5abd415db96605e2f486dc95 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:11.027892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03235e34-3882-4d1e-9975-4efe24367f11 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.038621 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03235e34-3882-4d1e-9975-4efe24367f11 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.061410 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:11.106317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.107119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.107407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.107717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] PATH_INFO: `/v3/users/b23499c05fbc4dca8c868798b4738946` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.109068 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23499c05fbc4dca8c868798b4738946)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:11.110963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:11.163493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23499c05fbc4dca8c868798b4738946'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.164378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23499c05fbc4dca8c868798b4738946'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.180029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23499c05fbc4dca8c868798b4738946'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.188635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23499c05fbc4dca8c868798b4738946'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.198110 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-549c0d25-2f46-4121-a4ac-1d1b718b2028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23499c05fbc4dca8c868798b4738946'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.207119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1316/2628] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:52:11 2022] DELETE /identity/v3/users/b23499c05fbc4dca8c868798b4738946 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:11.511420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.514812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:11.542621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79a284-a04e-4e6a-a575-62d5edf48ca5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.599522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:11.622406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.623198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.623506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.623786 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] PATH_INFO: `/v3/projects/7d3dab7b71ff4e8c86376b88f42fa837` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.625093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d3dab7b71ff4e8c86376b88f42fa837)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:11.627042 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:11.687495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d3dab7b71ff4e8c86376b88f42fa837', 'request_id': 'req-e40ffb3a-c432-4da5-983e-0c164cd47985'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.695181 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d3dab7b71ff4e8c86376b88f42fa837', 'request_id': 'req-e40ffb3a-c432-4da5-983e-0c164cd47985'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.703471 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d3dab7b71ff4e8c86376b88f42fa837', 'request_id': 'req-e40ffb3a-c432-4da5-983e-0c164cd47985'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.725876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] The token cache is being invalidate because project 7d3dab7b71ff4e8c86376b88f42fa837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:52:11.726422 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.727800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e40ffb3a-c432-4da5-983e-0c164cd47985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:11.729765 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1313/2629] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:52:11 2022] DELETE /identity/v3/projects/7d3dab7b71ff4e8c86376b88f42fa837 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:52:11.735112 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed90f27-96de-4699-bfa1-22c622984df4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.735920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed90f27-96de-4699-bfa1-22c622984df4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.735920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed90f27-96de-4699-bfa1-22c622984df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.756440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bed90f27-96de-4699-bfa1-22c622984df4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:11.757773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bed90f27-96de-4699-bfa1-22c622984df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:11.829558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1317/2630] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:52:11.840943 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c79c9-563e-4f31-b611-320420f328e7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.841505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c79c9-563e-4f31-b611-320420f328e7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.841794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c79c9-563e-4f31-b611-320420f328e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.862367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-188c79c9-563e-4f31-b611-320420f328e7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:11.863642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-188c79c9-563e-4f31-b611-320420f328e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:11.872176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1314/2631] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:52:11.877176 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed90f27-96de-4699-bfa1-22c622984df4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.888359 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed90f27-96de-4699-bfa1-22c622984df4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.893713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33d774-191c-4b77-b956-5b1da3970777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.894409 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33d774-191c-4b77-b956-5b1da3970777 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.894757 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33d774-191c-4b77-b956-5b1da3970777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.895041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33d774-191c-4b77-b956-5b1da3970777 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.895824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33d774-191c-4b77-b956-5b1da3970777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:11.896251 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33d774-191c-4b77-b956-5b1da3970777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:11.897905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33d774-191c-4b77-b956-5b1da3970777 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:11.911571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1318/2632] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:11.916560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c79c9-563e-4f31-b611-320420f328e7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.928236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c79c9-563e-4f31-b611-320420f328e7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.934030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f2d0-dcc5-4825-a706-131b50094088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.934792 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f2d0-dcc5-4825-a706-131b50094088 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.935229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f2d0-dcc5-4825-a706-131b50094088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.935442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f2d0-dcc5-4825-a706-131b50094088 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.936447 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6f2d0-dcc5-4825-a706-131b50094088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:11.938790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6f2d0-dcc5-4825-a706-131b50094088 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:11.973117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1315/2633] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:52:11 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:11.979505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33d774-191c-4b77-b956-5b1da3970777 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.990084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33d774-191c-4b77-b956-5b1da3970777 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.021624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:12.055556 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef82523-7b52-4907-bb08-c92474abc8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.056285 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef82523-7b52-4907-bb08-c92474abc8fe admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.056568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef82523-7b52-4907-bb08-c92474abc8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.056850 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef82523-7b52-4907-bb08-c92474abc8fe admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.057549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef82523-7b52-4907-bb08-c92474abc8fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.059518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef82523-7b52-4907-bb08-c92474abc8fe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.085600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1319/2634] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:52:11 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:12.090018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f2d0-dcc5-4825-a706-131b50094088 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.102067 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f2d0-dcc5-4825-a706-131b50094088 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.107027 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cad402-2dbe-4945-86a7-6464ea1bea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.107744 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cad402-2dbe-4945-86a7-6464ea1bea8f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.108034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cad402-2dbe-4945-86a7-6464ea1bea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.108347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cad402-2dbe-4945-86a7-6464ea1bea8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.109083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cad402-2dbe-4945-86a7-6464ea1bea8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.110754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cad402-2dbe-4945-86a7-6464ea1bea8f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.122725 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1316/2635] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:12.129225 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef82523-7b52-4907-bb08-c92474abc8fe admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.140391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef82523-7b52-4907-bb08-c92474abc8fe admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.145576 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29052b-29c2-4089-94ec-b987b73ddc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.146257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29052b-29c2-4089-94ec-b987b73ddc7a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.146560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29052b-29c2-4089-94ec-b987b73ddc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.146834 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29052b-29c2-4089-94ec-b987b73ddc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.147574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29052b-29c2-4089-94ec-b987b73ddc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.149336 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29052b-29c2-4089-94ec-b987b73ddc7a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.160921 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1320/2636] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:12.165711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cad402-2dbe-4945-86a7-6464ea1bea8f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.176796 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cad402-2dbe-4945-86a7-6464ea1bea8f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.184255 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694346-e6ef-4876-bf34-e3adca704094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.185395 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694346-e6ef-4876-bf34-e3adca704094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.185706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694346-e6ef-4876-bf34-e3adca704094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.185985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694346-e6ef-4876-bf34-e3adca704094 admin admin] PATH_INFO: `/v3/projects/9ec2873a671f4e37957498a03b8f8d3c/users/32d5915c2f774bbfbe14607c9810a521/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.204506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef694346-e6ef-4876-bf34-e3adca704094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec2873a671f4e37957498a03b8f8d3c, user_id=32d5915c2f774bbfbe14607c9810a521, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.207318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef694346-e6ef-4876-bf34-e3adca704094 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.224882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1317/2637] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:52:12 2022] PUT /identity/v3/projects/9ec2873a671f4e37957498a03b8f8d3c/users/32d5915c2f774bbfbe14607c9810a521/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:12.236652 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a26d1-f8da-493c-9768-7dde2506feba None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.236940 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a26d1-f8da-493c-9768-7dde2506feba None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.237245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a26d1-f8da-493c-9768-7dde2506feba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.267142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a27a26d1-f8da-493c-9768-7dde2506feba None None] MFA Rules not processed for user `32d5915c2f774bbfbe14607c9810a521`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:12.268420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a27a26d1-f8da-493c-9768-7dde2506feba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:12.341234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1321/2638] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:12.345921 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694346-e6ef-4876-bf34-e3adca704094 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.414240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694346-e6ef-4876-bf34-e3adca704094 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.419248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae53256-4782-4515-af28-f51ca7078a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.419967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae53256-4782-4515-af28-f51ca7078a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.420258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae53256-4782-4515-af28-f51ca7078a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.420533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae53256-4782-4515-af28-f51ca7078a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.421231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae53256-4782-4515-af28-f51ca7078a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.423144 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae53256-4782-4515-af28-f51ca7078a64 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.436920 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:12.452670 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1318/2639] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:52:12 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:12.457093 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a26d1-f8da-493c-9768-7dde2506feba None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.469146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a26d1-f8da-493c-9768-7dde2506feba None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.538146 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151225c6-dda8-44ef-83b0-9fbaae40b8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.539309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151225c6-dda8-44ef-83b0-9fbaae40b8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.539733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151225c6-dda8-44ef-83b0-9fbaae40b8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.540122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151225c6-dda8-44ef-83b0-9fbaae40b8f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.541256 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151225c6-dda8-44ef-83b0-9fbaae40b8f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.544341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151225c6-dda8-44ef-83b0-9fbaae40b8f8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.571475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1322/2640] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:52:12 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:52:12.576473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae53256-4782-4515-af28-f51ca7078a64 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.590103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae53256-4782-4515-af28-f51ca7078a64 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.599768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddec92-a815-4585-bb42-1873b544d9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.600817 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddec92-a815-4585-bb42-1873b544d9ed admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.601118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddec92-a815-4585-bb42-1873b544d9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.601521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddec92-a815-4585-bb42-1873b544d9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.602265 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddec92-a815-4585-bb42-1873b544d9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.604411 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddec92-a815-4585-bb42-1873b544d9ed admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.616311 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1319/2641] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:12.620739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151225c6-dda8-44ef-83b0-9fbaae40b8f8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.631902 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151225c6-dda8-44ef-83b0-9fbaae40b8f8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.638399 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb442de9-52d8-4d7f-b6eb-127eae935d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.639391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb442de9-52d8-4d7f-b6eb-127eae935d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.639711 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb442de9-52d8-4d7f-b6eb-127eae935d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.640178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb442de9-52d8-4d7f-b6eb-127eae935d4c admin admin] PATH_INFO: `/v3/projects/a88751892cad40329103069ec0082fc2/users/a7ceb6a4ea564db4a595cf43495e60de/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.656166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb442de9-52d8-4d7f-b6eb-127eae935d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88751892cad40329103069ec0082fc2, user_id=a7ceb6a4ea564db4a595cf43495e60de, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.658839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb442de9-52d8-4d7f-b6eb-127eae935d4c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.674351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1323/2642] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:52:12 2022] PUT /identity/v3/projects/a88751892cad40329103069ec0082fc2/users/a7ceb6a4ea564db4a595cf43495e60de/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:12.678631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddec92-a815-4585-bb42-1873b544d9ed admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.689334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddec92-a815-4585-bb42-1873b544d9ed admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.756457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f2a66-bb1d-4577-b67d-173d49701d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.757271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f2a66-bb1d-4577-b67d-173d49701d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.757678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f2a66-bb1d-4577-b67d-173d49701d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.757952 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f2a66-bb1d-4577-b67d-173d49701d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.758703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f2a66-bb1d-4577-b67d-173d49701d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.760405 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f2a66-bb1d-4577-b67d-173d49701d2d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.770923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1320/2643] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:12.775378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb442de9-52d8-4d7f-b6eb-127eae935d4c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.784106 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:12.787117 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb442de9-52d8-4d7f-b6eb-127eae935d4c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.794409 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a011f-d060-455e-a1d1-08df7476c4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.795322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a011f-d060-455e-a1d1-08df7476c4f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.795648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a011f-d060-455e-a1d1-08df7476c4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.795933 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a011f-d060-455e-a1d1-08df7476c4f1 admin admin] PATH_INFO: `/v3/projects/a88751892cad40329103069ec0082fc2/users/a7ceb6a4ea564db4a595cf43495e60de/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.797975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a011f-d060-455e-a1d1-08df7476c4f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88751892cad40329103069ec0082fc2, user_id=a7ceb6a4ea564db4a595cf43495e60de, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.800753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a011f-d060-455e-a1d1-08df7476c4f1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.812382 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1324/2644] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:52:12 2022] PUT /identity/v3/projects/a88751892cad40329103069ec0082fc2/users/a7ceb6a4ea564db4a595cf43495e60de/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:12.823282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45242476-9539-4981-87f7-f54381add219 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.823614 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45242476-9539-4981-87f7-f54381add219 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.823890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45242476-9539-4981-87f7-f54381add219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.853814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-45242476-9539-4981-87f7-f54381add219 None None] MFA Rules not processed for user `a7ceb6a4ea564db4a595cf43495e60de`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:12.855047 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-45242476-9539-4981-87f7-f54381add219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:12.929227 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1321/2645] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:12.944284 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a011f-d060-455e-a1d1-08df7476c4f1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.946624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-403a011f-d060-455e-a1d1-08df7476c4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:13.007574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a011f-d060-455e-a1d1-08df7476c4f1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.011156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e143e0-541a-40a5-8b90-55375708b133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.011851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e143e0-541a-40a5-8b90-55375708b133 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.012128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e143e0-541a-40a5-8b90-55375708b133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.012403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e143e0-541a-40a5-8b90-55375708b133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.073733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e143e0-541a-40a5-8b90-55375708b133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:13.075220 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e143e0-541a-40a5-8b90-55375708b133 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:13.085790 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1325/2646] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:22.925896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=1,keystone:SELECT=70 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:23.080656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=51,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:23.956584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45242476-9539-4981-87f7-f54381add219 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:23.967503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45242476-9539-4981-87f7-f54381add219 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:23.971755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d5b40-9e7e-4788-a343-d2fcf3dda49b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:23.972483 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d5b40-9e7e-4788-a343-d2fcf3dda49b service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:23.972775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d5b40-9e7e-4788-a343-d2fcf3dda49b service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:23.973044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d5b40-9e7e-4788-a343-d2fcf3dda49b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:23.983017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d5b40-9e7e-4788-a343-d2fcf3dda49b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:23.984525 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d5b40-9e7e-4788-a343-d2fcf3dda49b service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:23.994800 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1322/2647] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:32.331476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d8025-6341-4551-9448-976f1d7a144a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.331999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d8025-6341-4551-9448-976f1d7a144a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.331999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d8025-6341-4551-9448-976f1d7a144a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.350931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-747d8025-6341-4551-9448-976f1d7a144a None None] MFA Rules not processed for user `7cd67f4fdca84f328a7fe02623304f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:32.352177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-747d8025-6341-4551-9448-976f1d7a144a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:32.409680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1326/2648] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:32.983890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d5b40-9e7e-4788-a343-d2fcf3dda49b service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.986970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c57d5b40-9e7e-4788-a343-d2fcf3dda49b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:33.073194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d5b40-9e7e-4788-a343-d2fcf3dda49b service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:33.078241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45736167-8e2a-4651-b889-e1b93a87057f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:33.078852 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45736167-8e2a-4651-b889-e1b93a87057f service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:33.079141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45736167-8e2a-4651-b889-e1b93a87057f service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:33.079421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45736167-8e2a-4651-b889-e1b93a87057f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:33.081264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-45736167-8e2a-4651-b889-e1b93a87057f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:33.092668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45736167-8e2a-4651-b889-e1b93a87057f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:33.094262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45736167-8e2a-4651-b889-e1b93a87057f service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:33.104375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1323/2649] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:38.086645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d8025-6341-4551-9448-976f1d7a144a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:38.088776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-747d8025-6341-4551-9448-976f1d7a144a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:38.145755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d8025-6341-4551-9448-976f1d7a144a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:38.149378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0db49e-33a1-48cf-83fc-fefefbcf254b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:38.150051 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0db49e-33a1-48cf-83fc-fefefbcf254b service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:38.150341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0db49e-33a1-48cf-83fc-fefefbcf254b service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:38.150623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0db49e-33a1-48cf-83fc-fefefbcf254b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:38.153345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8e0db49e-33a1-48cf-83fc-fefefbcf254b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:38.219379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0db49e-33a1-48cf-83fc-fefefbcf254b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:38.220895 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0db49e-33a1-48cf-83fc-fefefbcf254b service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:38.230914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1327/2650] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:38.388977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ae3ad-c7c9-4d93-9739-976273f3c45c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:38.389486 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ae3ad-c7c9-4d93-9739-976273f3c45c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:38.389598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ae3ad-c7c9-4d93-9739-976273f3c45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:38.411120 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1d1ae3ad-c7c9-4d93-9739-976273f3c45c None None] MFA Rules not processed for user `4563cbf8fdbc492fae93bf92a004d0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:38.412625 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1d1ae3ad-c7c9-4d93-9739-976273f3c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:38.504002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1324/2651] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:39.514002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0db49e-33a1-48cf-83fc-fefefbcf254b service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:39.525092 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0db49e-33a1-48cf-83fc-fefefbcf254b service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:39.529937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabd0b5-401f-41ab-9495-5a7da21eeec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:39.530599 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabd0b5-401f-41ab-9495-5a7da21eeec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:39.530907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabd0b5-401f-41ab-9495-5a7da21eeec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:39.531182 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabd0b5-401f-41ab-9495-5a7da21eeec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:39.533294 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7eabd0b5-401f-41ab-9495-5a7da21eeec6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:39.544016 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabd0b5-401f-41ab-9495-5a7da21eeec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:39.545475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabd0b5-401f-41ab-9495-5a7da21eeec6 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:39.554466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1328/2652] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:42.494178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ae3ad-c7c9-4d93-9739-976273f3c45c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.496753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1d1ae3ad-c7c9-4d93-9739-976273f3c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:42.512554 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ae3ad-c7c9-4d93-9739-976273f3c45c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:42.518103 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:42.519019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.519298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.522638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] PATH_INFO: `/v3/users/c54ec6918ad44a31ba7137abb56b89b8` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.537597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c54ec6918ad44a31ba7137abb56b89b8)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:42.539602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:42.590428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c54ec6918ad44a31ba7137abb56b89b8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:42.591010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c54ec6918ad44a31ba7137abb56b89b8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:42.603426 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c54ec6918ad44a31ba7137abb56b89b8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:42.604597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:42.612161 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c54ec6918ad44a31ba7137abb56b89b8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:42.612862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5ca38058-79e9-479b-b722-2aba9c932654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c54ec6918ad44a31ba7137abb56b89b8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:42.626736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1325/2653] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:52:42 2022] DELETE /identity/v3/users/c54ec6918ad44a31ba7137abb56b89b8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:42.822479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabd0b5-401f-41ab-9495-5a7da21eeec6 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.834812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabd0b5-401f-41ab-9495-5a7da21eeec6 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:42.887885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:42.916923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:42.917936 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.918433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.918716 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] PATH_INFO: `/v3/projects/b29d4807663e4bfb934b2f7b3880f144` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.940746 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29d4807663e4bfb934b2f7b3880f144)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:42.946491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.001656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29d4807663e4bfb934b2f7b3880f144', 'request_id': 'req-f0089ebd-c700-471b-9d2d-8abcd95d8445'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:43.015440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29d4807663e4bfb934b2f7b3880f144', 'request_id': 'req-f0089ebd-c700-471b-9d2d-8abcd95d8445'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:43.022067 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29d4807663e4bfb934b2f7b3880f144', 'request_id': 'req-f0089ebd-c700-471b-9d2d-8abcd95d8445'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:43.035851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] The token cache is being invalidate because project b29d4807663e4bfb934b2f7b3880f144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:52:43.036179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:43.036904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0089ebd-c700-471b-9d2d-8abcd95d8445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:43.038446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1329/2654] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:52:42 2022] DELETE /identity/v3/projects/b29d4807663e4bfb934b2f7b3880f144 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:43.043598 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35822c9b-b947-46ea-8c17-9b5b61fbcb01 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.043892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35822c9b-b947-46ea-8c17-9b5b61fbcb01 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.044174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35822c9b-b947-46ea-8c17-9b5b61fbcb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.064157 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-35822c9b-b947-46ea-8c17-9b5b61fbcb01 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:43.065406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-35822c9b-b947-46ea-8c17-9b5b61fbcb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:43.126335 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1326/2655] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:43.134219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8863d0d-8503-4616-b178-44019136261c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.134673 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8863d0d-8503-4616-b178-44019136261c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.135029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8863d0d-8503-4616-b178-44019136261c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.155927 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c8863d0d-8503-4616-b178-44019136261c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:43.157253 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c8863d0d-8503-4616-b178-44019136261c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:43.165972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1330/2656] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:52:43.170586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35822c9b-b947-46ea-8c17-9b5b61fbcb01 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.182257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35822c9b-b947-46ea-8c17-9b5b61fbcb01 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.188255 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e220b-138c-44e2-89ec-d46296466b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.189410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e220b-138c-44e2-89ec-d46296466b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.189704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e220b-138c-44e2-89ec-d46296466b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.189996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e220b-138c-44e2-89ec-d46296466b97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.190734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e220b-138c-44e2-89ec-d46296466b97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:43.191050 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e220b-138c-44e2-89ec-d46296466b97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.193034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e220b-138c-44e2-89ec-d46296466b97 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.207498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1327/2657] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:43.211803 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8863d0d-8503-4616-b178-44019136261c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.221292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8863d0d-8503-4616-b178-44019136261c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.225962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d719-b34e-436c-8dbb-151f116446f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.226618 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d719-b34e-436c-8dbb-151f116446f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.226901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d719-b34e-436c-8dbb-151f116446f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.227174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d719-b34e-436c-8dbb-151f116446f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.227909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d719-b34e-436c-8dbb-151f116446f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.229818 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d719-b34e-436c-8dbb-151f116446f1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.257543 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1331/2658] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:52:43 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:43.263179 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e220b-138c-44e2-89ec-d46296466b97 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.273757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e220b-138c-44e2-89ec-d46296466b97 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.334277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77d080-53a9-481f-a767-cd000f381d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.334960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77d080-53a9-481f-a767-cd000f381d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.335250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77d080-53a9-481f-a767-cd000f381d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.335570 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77d080-53a9-481f-a767-cd000f381d05 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.336275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77d080-53a9-481f-a767-cd000f381d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.338213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77d080-53a9-481f-a767-cd000f381d05 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.360324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1328/2659] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:52:43 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:52:43.364773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d719-b34e-436c-8dbb-151f116446f1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.375293 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d719-b34e-436c-8dbb-151f116446f1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.380946 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc802d-48f4-4088-bba9-d4e24c5de005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.381553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc802d-48f4-4088-bba9-d4e24c5de005 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.381833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc802d-48f4-4088-bba9-d4e24c5de005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.382103 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc802d-48f4-4088-bba9-d4e24c5de005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.382821 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc802d-48f4-4088-bba9-d4e24c5de005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.384475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc802d-48f4-4088-bba9-d4e24c5de005 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.395502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1332/2660] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:52:43.399373 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77d080-53a9-481f-a767-cd000f381d05 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.410003 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77d080-53a9-481f-a767-cd000f381d05 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.415369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94f222-2d68-41d0-8e60-fb1d15fef071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.416121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94f222-2d68-41d0-8e60-fb1d15fef071 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.416407 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94f222-2d68-41d0-8e60-fb1d15fef071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.416683 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94f222-2d68-41d0-8e60-fb1d15fef071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.417957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94f222-2d68-41d0-8e60-fb1d15fef071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.419560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94f222-2d68-41d0-8e60-fb1d15fef071 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.431247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1329/2661] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:43.437350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc802d-48f4-4088-bba9-d4e24c5de005 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.447997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc802d-48f4-4088-bba9-d4e24c5de005 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.453857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499cb8b-1e76-4e82-a3eb-223ffbbefab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.454745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499cb8b-1e76-4e82-a3eb-223ffbbefab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.455027 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499cb8b-1e76-4e82-a3eb-223ffbbefab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.455300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499cb8b-1e76-4e82-a3eb-223ffbbefab8 admin admin] PATH_INFO: `/v3/projects/a4263b1caffd421a979fa7d4c0686bd6/users/90ff7162fe3a4bffaacb6e33f1870702/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.469914 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499cb8b-1e76-4e82-a3eb-223ffbbefab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4263b1caffd421a979fa7d4c0686bd6, user_id=90ff7162fe3a4bffaacb6e33f1870702, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.472603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499cb8b-1e76-4e82-a3eb-223ffbbefab8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.486608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1333/2662] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:52:43 2022] PUT /identity/v3/projects/a4263b1caffd421a979fa7d4c0686bd6/users/90ff7162fe3a4bffaacb6e33f1870702/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:43.497912 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4ef54-091f-45b7-bfef-d6820aa073c4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.498457 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4ef54-091f-45b7-bfef-d6820aa073c4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.498564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4ef54-091f-45b7-bfef-d6820aa073c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.532738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0ce4ef54-091f-45b7-bfef-d6820aa073c4 None None] MFA Rules not processed for user `90ff7162fe3a4bffaacb6e33f1870702`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:43.533980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0ce4ef54-091f-45b7-bfef-d6820aa073c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:43.579319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:43.607192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1330/2663] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:43 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:43.611841 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499cb8b-1e76-4e82-a3eb-223ffbbefab8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.676165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499cb8b-1e76-4e82-a3eb-223ffbbefab8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.682367 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31176ed-e140-4aad-a419-fe319f6caed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.683074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31176ed-e140-4aad-a419-fe319f6caed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.683351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31176ed-e140-4aad-a419-fe319f6caed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.683662 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31176ed-e140-4aad-a419-fe319f6caed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.684357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31176ed-e140-4aad-a419-fe319f6caed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.686307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31176ed-e140-4aad-a419-fe319f6caed0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.690205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:43.716798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1334/2664] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:52:43 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:43.724044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4ef54-091f-45b7-bfef-d6820aa073c4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.736450 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4ef54-091f-45b7-bfef-d6820aa073c4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.801038 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6b574-86b4-467a-8151-9f2a9ec468f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.801372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6b574-86b4-467a-8151-9f2a9ec468f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.801662 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6b574-86b4-467a-8151-9f2a9ec468f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.801968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6b574-86b4-467a-8151-9f2a9ec468f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.802691 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6b574-86b4-467a-8151-9f2a9ec468f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.804754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6b574-86b4-467a-8151-9f2a9ec468f1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.827634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1331/2665] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:52:43 2022] POST /identity/v3/users => generated 309 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:43.831567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31176ed-e140-4aad-a419-fe319f6caed0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.842236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31176ed-e140-4aad-a419-fe319f6caed0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.846834 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095520d-57c1-4088-9f18-2c9e63fb97e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.847569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095520d-57c1-4088-9f18-2c9e63fb97e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.847873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095520d-57c1-4088-9f18-2c9e63fb97e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.848158 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095520d-57c1-4088-9f18-2c9e63fb97e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.849107 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095520d-57c1-4088-9f18-2c9e63fb97e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.850773 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095520d-57c1-4088-9f18-2c9e63fb97e5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.990981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1335/2666] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:43.992228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6b574-86b4-467a-8151-9f2a9ec468f1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.002948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6b574-86b4-467a-8151-9f2a9ec468f1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.008283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240444bf-edaf-4892-9d84-3b42452edd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.009196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240444bf-edaf-4892-9d84-3b42452edd1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.009479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240444bf-edaf-4892-9d84-3b42452edd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.009755 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240444bf-edaf-4892-9d84-3b42452edd1e admin admin] PATH_INFO: `/v3/projects/508b2f1f78e1417d9b346f749b3ba412/users/41212c540f0c497891eef33a68070fc2/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.025627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240444bf-edaf-4892-9d84-3b42452edd1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=508b2f1f78e1417d9b346f749b3ba412, user_id=41212c540f0c497891eef33a68070fc2, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.028332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240444bf-edaf-4892-9d84-3b42452edd1e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.044634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1332/2667] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:52:43 2022] PUT /identity/v3/projects/508b2f1f78e1417d9b346f749b3ba412/users/41212c540f0c497891eef33a68070fc2/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:44.048938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095520d-57c1-4088-9f18-2c9e63fb97e5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.060119 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095520d-57c1-4088-9f18-2c9e63fb97e5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.124346 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384eaf2-eadd-4ee4-a1aa-2569fea1d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.125048 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384eaf2-eadd-4ee4-a1aa-2569fea1d367 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.125329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384eaf2-eadd-4ee4-a1aa-2569fea1d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.125606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384eaf2-eadd-4ee4-a1aa-2569fea1d367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.126353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384eaf2-eadd-4ee4-a1aa-2569fea1d367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.128037 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384eaf2-eadd-4ee4-a1aa-2569fea1d367 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.138573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1336/2668] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:44.143567 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240444bf-edaf-4892-9d84-3b42452edd1e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.154030 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240444bf-edaf-4892-9d84-3b42452edd1e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.158564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec2181-9474-4507-a4ba-83195fb7804f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.159610 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec2181-9474-4507-a4ba-83195fb7804f admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.159898 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec2181-9474-4507-a4ba-83195fb7804f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.160176 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec2181-9474-4507-a4ba-83195fb7804f admin admin] PATH_INFO: `/v3/projects/508b2f1f78e1417d9b346f749b3ba412/users/41212c540f0c497891eef33a68070fc2/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.162264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec2181-9474-4507-a4ba-83195fb7804f admin admin] RBAC: Authorizing `identity:create_grant(project_id=508b2f1f78e1417d9b346f749b3ba412, user_id=41212c540f0c497891eef33a68070fc2, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.165283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec2181-9474-4507-a4ba-83195fb7804f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.176088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1333/2669] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:52:44 2022] PUT /identity/v3/projects/508b2f1f78e1417d9b346f749b3ba412/users/41212c540f0c497891eef33a68070fc2/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:44.186945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa0493-c96f-40e5-a468-8202bf3275d3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.187241 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa0493-c96f-40e5-a468-8202bf3275d3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.187551 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa0493-c96f-40e5-a468-8202bf3275d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.215147 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f5aa0493-c96f-40e5-a468-8202bf3275d3 None None] MFA Rules not processed for user `41212c540f0c497891eef33a68070fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:44.216394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f5aa0493-c96f-40e5-a468-8202bf3275d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:44.289885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1337/2670] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:44.300914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec2181-9474-4507-a4ba-83195fb7804f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.302868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-85ec2181-9474-4507-a4ba-83195fb7804f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:44.363118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec2181-9474-4507-a4ba-83195fb7804f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.367660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038e2d3-1722-4c41-b54c-4444ecf8d243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.368345 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038e2d3-1722-4c41-b54c-4444ecf8d243 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.368627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038e2d3-1722-4c41-b54c-4444ecf8d243 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.368902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038e2d3-1722-4c41-b54c-4444ecf8d243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.378443 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038e2d3-1722-4c41-b54c-4444ecf8d243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.379914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038e2d3-1722-4c41-b54c-4444ecf8d243 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.389564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1334/2671] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:44 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:44.538581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa0493-c96f-40e5-a468-8202bf3275d3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.598299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:44.605060 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa0493-c96f-40e5-a468-8202bf3275d3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.610343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.611222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.611532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.611806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] PATH_INFO: `/v3/users/90ff7162fe3a4bffaacb6e33f1870702` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.612990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=90ff7162fe3a4bffaacb6e33f1870702)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.614918 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.661871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90ff7162fe3a4bffaacb6e33f1870702'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:44.662520 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90ff7162fe3a4bffaacb6e33f1870702'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:44.674320 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90ff7162fe3a4bffaacb6e33f1870702'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:44.681045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90ff7162fe3a4bffaacb6e33f1870702'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:44.689863 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90ff7162fe3a4bffaacb6e33f1870702'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:44.695911 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1338/2672] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:52:44 2022] DELETE /identity/v3/users/90ff7162fe3a4bffaacb6e33f1870702 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:44.700410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ac079-5cf5-44de-8843-bc44c9e50c90 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.700871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ac079-5cf5-44de-8843-bc44c9e50c90 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.701095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ac079-5cf5-44de-8843-bc44c9e50c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.718123 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e99ac079-5cf5-44de-8843-bc44c9e50c90 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:44.719312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e99ac079-5cf5-44de-8843-bc44c9e50c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:44.748584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:44.791214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1335/2673] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:52:44.801314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.802996 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:44.863799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5ca0c-08fe-4ead-a377-f3f05a5cc7dc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.869082 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5ef13-28cd-4532-8a1b-34ebd5a2283d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.870194 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5ef13-28cd-4532-8a1b-34ebd5a2283d service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.870695 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5ef13-28cd-4532-8a1b-34ebd5a2283d service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.871170 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5ef13-28cd-4532-8a1b-34ebd5a2283d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.884501 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5ef13-28cd-4532-8a1b-34ebd5a2283d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.885957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5ef13-28cd-4532-8a1b-34ebd5a2283d service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.897536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1339/2674] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:45.169821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ac079-5cf5-44de-8843-bc44c9e50c90 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:45.171383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e99ac079-5cf5-44de-8843-bc44c9e50c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:45.185654 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ac079-5cf5-44de-8843-bc44c9e50c90 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:45.190968 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:45.192039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.192419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.192853 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] PATH_INFO: `/v3/projects/a4263b1caffd421a979fa7d4c0686bd6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.194190 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4263b1caffd421a979fa7d4c0686bd6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:45.197180 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:45.252220 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4263b1caffd421a979fa7d4c0686bd6', 'request_id': 'req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.260342 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4263b1caffd421a979fa7d4c0686bd6', 'request_id': 'req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.267150 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4263b1caffd421a979fa7d4c0686bd6', 'request_id': 'req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.291386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] The token cache is being invalidate because project a4263b1caffd421a979fa7d4c0686bd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:52:45.291386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.291386 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.292880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1336/2675] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:52:45 2022] DELETE /identity/v3/projects/a4263b1caffd421a979fa7d4c0686bd6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:45.297013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5ef13-28cd-4532-8a1b-34ebd5a2283d service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:45.307211 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5ef13-28cd-4532-8a1b-34ebd5a2283d service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:45.364169 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:45.365073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.365362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.365640 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] PATH_INFO: `/v3/users/41212c540f0c497891eef33a68070fc2` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.366816 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=41212c540f0c497891eef33a68070fc2)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:45.368859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:45.389163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:45.421995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41212c540f0c497891eef33a68070fc2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.422664 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41212c540f0c497891eef33a68070fc2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.434643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41212c540f0c497891eef33a68070fc2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.441653 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41212c540f0c497891eef33a68070fc2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.451954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7feae61f-8c6f-4693-92b7-dec18cdbad5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41212c540f0c497891eef33a68070fc2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.460976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1340/2676] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:52:45 2022] DELETE /identity/v3/users/41212c540f0c497891eef33a68070fc2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:45.746019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:45.748268 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:45.763396 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba91ea0-a507-42f3-bfba-7a3a0859f6a5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:45.834319 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:45.835199 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.835511 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.835790 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] PATH_INFO: `/v3/projects/508b2f1f78e1417d9b346f749b3ba412` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.836980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=508b2f1f78e1417d9b346f749b3ba412)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:45.838865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:45.885686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '508b2f1f78e1417d9b346f749b3ba412', 'request_id': 'req-9b2597d7-e3f4-475c-ae9c-62ac5020a535'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.891707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '508b2f1f78e1417d9b346f749b3ba412', 'request_id': 'req-9b2597d7-e3f4-475c-ae9c-62ac5020a535'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.895442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:45.898278 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '508b2f1f78e1417d9b346f749b3ba412', 'request_id': 'req-9b2597d7-e3f4-475c-ae9c-62ac5020a535'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.912489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] The token cache is being invalidate because project 508b2f1f78e1417d9b346f749b3ba412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:52:45.912775 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.913384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9b2597d7-e3f4-475c-ae9c-62ac5020a535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:45.914901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1337/2677] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:52:45 2022] DELETE /identity/v3/projects/508b2f1f78e1417d9b346f749b3ba412 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:45.920025 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f188968-4522-4fcb-bd11-99527338736f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.920507 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f188968-4522-4fcb-bd11-99527338736f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.920607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f188968-4522-4fcb-bd11-99527338736f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.939975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8f188968-4522-4fcb-bd11-99527338736f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:45.941517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8f188968-4522-4fcb-bd11-99527338736f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:46.005270 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1341/2678] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:46.013201 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7d845-ebaf-4e4e-97b8-319486e2bd28 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.013490 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7d845-ebaf-4e4e-97b8-319486e2bd28 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.013769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7d845-ebaf-4e4e-97b8-319486e2bd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.037759 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-81a7d845-ebaf-4e4e-97b8-319486e2bd28 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:46.038903 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-81a7d845-ebaf-4e4e-97b8-319486e2bd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:46.047967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1338/2679] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:46.052452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f188968-4522-4fcb-bd11-99527338736f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.063055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f188968-4522-4fcb-bd11-99527338736f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.068677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b05a88-2b62-41f8-bdec-6237f4026c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.069348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b05a88-2b62-41f8-bdec-6237f4026c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.069664 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b05a88-2b62-41f8-bdec-6237f4026c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.069928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b05a88-2b62-41f8-bdec-6237f4026c7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.070661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b05a88-2b62-41f8-bdec-6237f4026c7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:46.070971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b05a88-2b62-41f8-bdec-6237f4026c7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.072635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b05a88-2b62-41f8-bdec-6237f4026c7a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.084894 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1342/2680] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:46.089513 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7d845-ebaf-4e4e-97b8-319486e2bd28 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.099046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7d845-ebaf-4e4e-97b8-319486e2bd28 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.105397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db90d54-b261-4bda-a0da-d5b50ec47199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.106049 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db90d54-b261-4bda-a0da-d5b50ec47199 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.106324 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db90d54-b261-4bda-a0da-d5b50ec47199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.107019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db90d54-b261-4bda-a0da-d5b50ec47199 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.107291 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db90d54-b261-4bda-a0da-d5b50ec47199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.109275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db90d54-b261-4bda-a0da-d5b50ec47199 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.142544 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1339/2681] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:52:46 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:46.146893 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b05a88-2b62-41f8-bdec-6237f4026c7a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.157669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b05a88-2b62-41f8-bdec-6237f4026c7a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.218969 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9a019-d3a9-4370-9cf6-282cfbd6234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.219683 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9a019-d3a9-4370-9cf6-282cfbd6234e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.219973 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9a019-d3a9-4370-9cf6-282cfbd6234e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.220254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9a019-d3a9-4370-9cf6-282cfbd6234e admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.220947 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9a019-d3a9-4370-9cf6-282cfbd6234e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.222913 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9a019-d3a9-4370-9cf6-282cfbd6234e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.243774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1343/2682] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:52:46 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:46.249992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db90d54-b261-4bda-a0da-d5b50ec47199 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.260607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db90d54-b261-4bda-a0da-d5b50ec47199 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.265695 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da07831-1ebc-42d5-8fde-192b5639b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.266358 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da07831-1ebc-42d5-8fde-192b5639b7bd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.266645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da07831-1ebc-42d5-8fde-192b5639b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.266929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da07831-1ebc-42d5-8fde-192b5639b7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.267694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da07831-1ebc-42d5-8fde-192b5639b7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.269297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da07831-1ebc-42d5-8fde-192b5639b7bd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.280809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1340/2683] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:46.284862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9a019-d3a9-4370-9cf6-282cfbd6234e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.295544 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9a019-d3a9-4370-9cf6-282cfbd6234e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.299847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b5976-328b-460a-8116-906a663fbe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.300505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b5976-328b-460a-8116-906a663fbe9b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.300794 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b5976-328b-460a-8116-906a663fbe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.301075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b5976-328b-460a-8116-906a663fbe9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.301799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b5976-328b-460a-8116-906a663fbe9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.303733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b5976-328b-460a-8116-906a663fbe9b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.318099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1344/2684] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:46.322277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da07831-1ebc-42d5-8fde-192b5639b7bd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.332402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da07831-1ebc-42d5-8fde-192b5639b7bd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.337738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ca785-663c-4d75-8fb4-e0c9e1896fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.338522 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ca785-663c-4d75-8fb4-e0c9e1896fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.338842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ca785-663c-4d75-8fb4-e0c9e1896fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.339264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ca785-663c-4d75-8fb4-e0c9e1896fe4 admin admin] PATH_INFO: `/v3/projects/897eab7ebd3940d483608d9a780a1611/users/da1bff9630f344f091286a40ae0e9b23/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.354459 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ca785-663c-4d75-8fb4-e0c9e1896fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897eab7ebd3940d483608d9a780a1611, user_id=da1bff9630f344f091286a40ae0e9b23, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.357153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ca785-663c-4d75-8fb4-e0c9e1896fe4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.372055 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1341/2685] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:52:46 2022] PUT /identity/v3/projects/897eab7ebd3940d483608d9a780a1611/users/da1bff9630f344f091286a40ae0e9b23/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:46.383872 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30a4ab-10c9-49af-9df0-dd8beaf9426d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.384179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30a4ab-10c9-49af-9df0-dd8beaf9426d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.384455 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30a4ab-10c9-49af-9df0-dd8beaf9426d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.390616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.413865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2f30a4ab-10c9-49af-9df0-dd8beaf9426d None None] MFA Rules not processed for user `da1bff9630f344f091286a40ae0e9b23`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:46.415090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2f30a4ab-10c9-49af-9df0-dd8beaf9426d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:46.481361 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1345/2686] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:46.486316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ca785-663c-4d75-8fb4-e0c9e1896fe4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.547034 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ca785-663c-4d75-8fb4-e0c9e1896fe4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.551914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c06a02-a050-45e1-9373-6ce98882540a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.552616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c06a02-a050-45e1-9373-6ce98882540a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.552938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c06a02-a050-45e1-9373-6ce98882540a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.553214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c06a02-a050-45e1-9373-6ce98882540a admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.553908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c06a02-a050-45e1-9373-6ce98882540a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.555890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c06a02-a050-45e1-9373-6ce98882540a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.585673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1342/2687] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:52:46 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:52:46.590370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30a4ab-10c9-49af-9df0-dd8beaf9426d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.601366 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30a4ab-10c9-49af-9df0-dd8beaf9426d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.658753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46dd79-558a-487c-85c5-a61253e5ca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.659439 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46dd79-558a-487c-85c5-a61253e5ca12 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.659760 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46dd79-558a-487c-85c5-a61253e5ca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.660047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46dd79-558a-487c-85c5-a61253e5ca12 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.660759 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46dd79-558a-487c-85c5-a61253e5ca12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.662796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46dd79-558a-487c-85c5-a61253e5ca12 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.685315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1346/2688] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:52:46 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:46.689998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c06a02-a050-45e1-9373-6ce98882540a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.700530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c06a02-a050-45e1-9373-6ce98882540a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.706275 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e06e88-f0b3-426b-8501-523684305ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.706945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e06e88-f0b3-426b-8501-523684305ced admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.707232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e06e88-f0b3-426b-8501-523684305ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.707545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e06e88-f0b3-426b-8501-523684305ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.708364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e06e88-f0b3-426b-8501-523684305ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.709974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e06e88-f0b3-426b-8501-523684305ced admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.721054 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1343/2689] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:46.725487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46dd79-558a-487c-85c5-a61253e5ca12 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.735443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46dd79-558a-487c-85c5-a61253e5ca12 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.740654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67498f01-2648-446c-9874-c081369ce4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.741550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67498f01-2648-446c-9874-c081369ce4e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.741836 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67498f01-2648-446c-9874-c081369ce4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.742118 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67498f01-2648-446c-9874-c081369ce4e4 admin admin] PATH_INFO: `/v3/projects/ba4cbee219a2447cb854c85b3910e8a4/users/9c0913cf66114b6794e1ebcc8cf42e39/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.757394 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67498f01-2648-446c-9874-c081369ce4e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4cbee219a2447cb854c85b3910e8a4, user_id=9c0913cf66114b6794e1ebcc8cf42e39, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.760249 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67498f01-2648-446c-9874-c081369ce4e4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.775464 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1347/2690] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:52:46 2022] PUT /identity/v3/projects/ba4cbee219a2447cb854c85b3910e8a4/users/9c0913cf66114b6794e1ebcc8cf42e39/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:46.781689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e06e88-f0b3-426b-8501-523684305ced admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.793279 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e06e88-f0b3-426b-8501-523684305ced admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.845657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.852779 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced80a3-d1ec-4f00-bfe8-d179e452f216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.853498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced80a3-d1ec-4f00-bfe8-d179e452f216 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.854155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced80a3-d1ec-4f00-bfe8-d179e452f216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.854821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced80a3-d1ec-4f00-bfe8-d179e452f216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.855591 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced80a3-d1ec-4f00-bfe8-d179e452f216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.857856 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced80a3-d1ec-4f00-bfe8-d179e452f216 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.868605 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1344/2691] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:46.872826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67498f01-2648-446c-9874-c081369ce4e4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.883197 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67498f01-2648-446c-9874-c081369ce4e4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.888199 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5111d-9486-4877-a015-266f5a5aee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.889094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5111d-9486-4877-a015-266f5a5aee96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.889378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5111d-9486-4877-a015-266f5a5aee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.889635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5111d-9486-4877-a015-266f5a5aee96 admin admin] PATH_INFO: `/v3/projects/ba4cbee219a2447cb854c85b3910e8a4/users/9c0913cf66114b6794e1ebcc8cf42e39/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.891677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5111d-9486-4877-a015-266f5a5aee96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4cbee219a2447cb854c85b3910e8a4, user_id=9c0913cf66114b6794e1ebcc8cf42e39, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.894313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5111d-9486-4877-a015-266f5a5aee96 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.906535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1348/2692] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:52:46 2022] PUT /identity/v3/projects/ba4cbee219a2447cb854c85b3910e8a4/users/9c0913cf66114b6794e1ebcc8cf42e39/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:46.917174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68acc-afdb-49df-a674-5509d61e1dcb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.917468 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68acc-afdb-49df-a674-5509d61e1dcb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.917747 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68acc-afdb-49df-a674-5509d61e1dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.948862 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a0f68acc-afdb-49df-a674-5509d61e1dcb None None] MFA Rules not processed for user `9c0913cf66114b6794e1ebcc8cf42e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:46.950073 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a0f68acc-afdb-49df-a674-5509d61e1dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:47.019802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1345/2693] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:47.031573 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5111d-9486-4877-a015-266f5a5aee96 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:47.033656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d3f5111d-9486-4877-a015-266f5a5aee96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:47.082013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:47.094438 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5111d-9486-4877-a015-266f5a5aee96 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:47.098424 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fa4a7-bf59-4202-a49b-aed649005e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:47.099104 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fa4a7-bf59-4202-a49b-aed649005e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.099392 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fa4a7-bf59-4202-a49b-aed649005e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.099706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fa4a7-bf59-4202-a49b-aed649005e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.109309 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fa4a7-bf59-4202-a49b-aed649005e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:47.110798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fa4a7-bf59-4202-a49b-aed649005e22 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:47.120583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1349/2694] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:47 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:47.194267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f68acc-afdb-49df-a674-5509d61e1dcb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:47.204848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f68acc-afdb-49df-a674-5509d61e1dcb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:47.209224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bd978-d4c6-4fc4-a09d-ecb0ac1159bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:47.209869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bd978-d4c6-4fc4-a09d-ecb0ac1159bd service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.210147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bd978-d4c6-4fc4-a09d-ecb0ac1159bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.210418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bd978-d4c6-4fc4-a09d-ecb0ac1159bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.267153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bd978-d4c6-4fc4-a09d-ecb0ac1159bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:47.268613 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bd978-d4c6-4fc4-a09d-ecb0ac1159bd service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:47.277367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1346/2695] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:52.325978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fa4a7-bf59-4202-a49b-aed649005e22 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:52.328472 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-295fa4a7-bf59-4202-a49b-aed649005e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:52.438386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fa4a7-bf59-4202-a49b-aed649005e22 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:52.445239 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72735d58-59b8-4238-af00-b3c072709b58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:52.446001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72735d58-59b8-4238-af00-b3c072709b58 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.446360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72735d58-59b8-4238-af00-b3c072709b58 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.446700 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72735d58-59b8-4238-af00-b3c072709b58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.448967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-72735d58-59b8-4238-af00-b3c072709b58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:52.468982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72735d58-59b8-4238-af00-b3c072709b58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:52.472379 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72735d58-59b8-4238-af00-b3c072709b58 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:52.486296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1350/2696] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:57.273546 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=65 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:58.468312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bd978-d4c6-4fc4-a09d-ecb0ac1159bd service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:58.469936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-458bd978-d4c6-4fc4-a09d-ecb0ac1159bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:58.538141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bd978-d4c6-4fc4-a09d-ecb0ac1159bd service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:58.543478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:58.544439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:58.544746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:58.545051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] PATH_INFO: `/v3/users/32d5915c2f774bbfbe14607c9810a521` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:58.546442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32d5915c2f774bbfbe14607c9810a521)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:58.548412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:58.599348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32d5915c2f774bbfbe14607c9810a521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:58.600052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32d5915c2f774bbfbe14607c9810a521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:58.612809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32d5915c2f774bbfbe14607c9810a521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:58.620333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32d5915c2f774bbfbe14607c9810a521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:58.621076 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32d5915c2f774bbfbe14607c9810a521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:58.639597 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1347/2697] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:52:58 2022] DELETE /identity/v3/users/32d5915c2f774bbfbe14607c9810a521 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:58.644177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2189227-c89b-452b-bd41-be1882f51dcb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:58.644641 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2189227-c89b-452b-bd41-be1882f51dcb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:58.644756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2189227-c89b-452b-bd41-be1882f51dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:58.662590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c2189227-c89b-452b-bd41-be1882f51dcb None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:58.663827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c2189227-c89b-452b-bd41-be1882f51dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:58.724404 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1351/2698] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:52:58.734316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:58.736026 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:58.799908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa07535-df4f-46a9-b692-2145aa9d79c9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:58.803620 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852452d5-dda7-463b-a47e-4cbed25917e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:58.804282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852452d5-dda7-463b-a47e-4cbed25917e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:58.804569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852452d5-dda7-463b-a47e-4cbed25917e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:58.804848 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852452d5-dda7-463b-a47e-4cbed25917e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:58.816196 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852452d5-dda7-463b-a47e-4cbed25917e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:58.817770 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852452d5-dda7-463b-a47e-4cbed25917e0 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:58.828204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1348/2699] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:52:59.102450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2189227-c89b-452b-bd41-be1882f51dcb None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:59.104030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c2189227-c89b-452b-bd41-be1882f51dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:59.124062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2189227-c89b-452b-bd41-be1882f51dcb None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:59.138342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:59.147592 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.148052 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.148449 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] PATH_INFO: `/v3/projects/9ec2873a671f4e37957498a03b8f8d3c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.151084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec2873a671f4e37957498a03b8f8d3c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:59.154678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:59.222886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec2873a671f4e37957498a03b8f8d3c', 'request_id': 'req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.240983 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec2873a671f4e37957498a03b8f8d3c', 'request_id': 'req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.244887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec2873a671f4e37957498a03b8f8d3c', 'request_id': 'req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.249100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:59.263865 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] The token cache is being invalidate because project 9ec2873a671f4e37957498a03b8f8d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:52:59.264419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.265427 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.267500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1352/2700] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:52:59 2022] DELETE /identity/v3/projects/9ec2873a671f4e37957498a03b8f8d3c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:59.271846 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852452d5-dda7-463b-a47e-4cbed25917e0 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:59.288134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852452d5-dda7-463b-a47e-4cbed25917e0 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:59.321933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:59.368298 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:59.369328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.369680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.370045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] PATH_INFO: `/v3/users/a7ceb6a4ea564db4a595cf43495e60de` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.371402 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ceb6a4ea564db4a595cf43495e60de)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:59.383378 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:59.424312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ceb6a4ea564db4a595cf43495e60de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.425346 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ceb6a4ea564db4a595cf43495e60de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.441297 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ceb6a4ea564db4a595cf43495e60de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.449314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ceb6a4ea564db4a595cf43495e60de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.450146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7e5f2545-0e4e-4a63-b057-a4f94375823c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ceb6a4ea564db4a595cf43495e60de'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.470703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1349/2701] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:52:59 2022] DELETE /identity/v3/users/a7ceb6a4ea564db4a595cf43495e60de => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:59.754997 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:59.756613 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:59.774461 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb108b-62b1-4c2f-91d9-2a2dc7e3ad9c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:59.838064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:59.838961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.839243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.839552 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] PATH_INFO: `/v3/projects/a88751892cad40329103069ec0082fc2` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.840934 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a88751892cad40329103069ec0082fc2)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:59.842981 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:59.898622 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a88751892cad40329103069ec0082fc2', 'request_id': 'req-8a6ba9da-123a-4546-8a47-cc7834a64ca5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.906560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a88751892cad40329103069ec0082fc2', 'request_id': 'req-8a6ba9da-123a-4546-8a47-cc7834a64ca5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.912607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a88751892cad40329103069ec0082fc2', 'request_id': 'req-8a6ba9da-123a-4546-8a47-cc7834a64ca5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.924967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] The token cache is being invalidate because project a88751892cad40329103069ec0082fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:52:59.925305 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.925976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8a6ba9da-123a-4546-8a47-cc7834a64ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:52:59.927864 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1353/2702] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:52:59 2022] DELETE /identity/v3/projects/a88751892cad40329103069ec0082fc2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:59.933858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f917-aeea-480b-80d5-c6c0ac1daba8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.933858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f917-aeea-480b-80d5-c6c0ac1daba8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.934114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f917-aeea-480b-80d5-c6c0ac1daba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.955673 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-68a8f917-aeea-480b-80d5-c6c0ac1daba8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:59.957039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-68a8f917-aeea-480b-80d5-c6c0ac1daba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:00.024092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1350/2703] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:00.032831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b1539-c1b7-40d0-8fde-e418b9c00242 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.032979 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b1539-c1b7-40d0-8fde-e418b9c00242 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.036668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b1539-c1b7-40d0-8fde-e418b9c00242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.058767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b53b1539-c1b7-40d0-8fde-e418b9c00242 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:00.060057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b53b1539-c1b7-40d0-8fde-e418b9c00242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:00.068650 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1354/2704] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:00.072991 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f917-aeea-480b-80d5-c6c0ac1daba8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.082516 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f917-aeea-480b-80d5-c6c0ac1daba8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.087099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9005-0674-4e4b-a56f-c72bf5d2edd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.087868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9005-0674-4e4b-a56f-c72bf5d2edd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.088156 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9005-0674-4e4b-a56f-c72bf5d2edd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.088432 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9005-0674-4e4b-a56f-c72bf5d2edd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.089160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9005-0674-4e4b-a56f-c72bf5d2edd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:00.089467 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9005-0674-4e4b-a56f-c72bf5d2edd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.091075 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9005-0674-4e4b-a56f-c72bf5d2edd2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.104018 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1351/2705] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:00.108274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b1539-c1b7-40d0-8fde-e418b9c00242 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.118149 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b1539-c1b7-40d0-8fde-e418b9c00242 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.122971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005d0bc-83d8-4e7b-941a-0b9be1bc645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.123676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005d0bc-83d8-4e7b-941a-0b9be1bc645e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.123972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005d0bc-83d8-4e7b-941a-0b9be1bc645e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.124253 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005d0bc-83d8-4e7b-941a-0b9be1bc645e admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.124962 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005d0bc-83d8-4e7b-941a-0b9be1bc645e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.126917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005d0bc-83d8-4e7b-941a-0b9be1bc645e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.153297 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1355/2706] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:00 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:00.157433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9005-0674-4e4b-a56f-c72bf5d2edd2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.167687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9005-0674-4e4b-a56f-c72bf5d2edd2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.197992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.231241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c6163-1d59-4bd6-b99c-5541fd8e8de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.231963 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c6163-1d59-4bd6-b99c-5541fd8e8de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.232250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c6163-1d59-4bd6-b99c-5541fd8e8de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.232532 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c6163-1d59-4bd6-b99c-5541fd8e8de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.233225 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c6163-1d59-4bd6-b99c-5541fd8e8de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.235141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c6163-1d59-4bd6-b99c-5541fd8e8de0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.254666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1352/2707] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:00 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:00.258856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005d0bc-83d8-4e7b-941a-0b9be1bc645e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.269105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005d0bc-83d8-4e7b-941a-0b9be1bc645e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.273655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1a705-de76-4d02-a013-793012eb195b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.274289 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1a705-de76-4d02-a013-793012eb195b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.274578 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1a705-de76-4d02-a013-793012eb195b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.274857 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1a705-de76-4d02-a013-793012eb195b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.275619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1a705-de76-4d02-a013-793012eb195b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.277236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1a705-de76-4d02-a013-793012eb195b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.288065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1356/2708] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:00.292099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c6163-1d59-4bd6-b99c-5541fd8e8de0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.302927 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c6163-1d59-4bd6-b99c-5541fd8e8de0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.307893 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2d8ab-be07-4f4e-ba04-55eee98ca4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.308662 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2d8ab-be07-4f4e-ba04-55eee98ca4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.308867 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2d8ab-be07-4f4e-ba04-55eee98ca4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.309197 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2d8ab-be07-4f4e-ba04-55eee98ca4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.310016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2d8ab-be07-4f4e-ba04-55eee98ca4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.311697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2d8ab-be07-4f4e-ba04-55eee98ca4d9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.322431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1353/2709] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:00.327091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1a705-de76-4d02-a013-793012eb195b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.337476 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1a705-de76-4d02-a013-793012eb195b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.342418 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cf7d7-6e77-4632-8a51-b0fc88feb9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.343321 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cf7d7-6e77-4632-8a51-b0fc88feb9c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.343643 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cf7d7-6e77-4632-8a51-b0fc88feb9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.343926 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cf7d7-6e77-4632-8a51-b0fc88feb9c9 admin admin] PATH_INFO: `/v3/projects/9261f18b1cd84ccbbfb97f028b3318c0/users/87d996f101874598992142bbfd6f3b5c/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.359162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cf7d7-6e77-4632-8a51-b0fc88feb9c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9261f18b1cd84ccbbfb97f028b3318c0, user_id=87d996f101874598992142bbfd6f3b5c, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.361851 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cf7d7-6e77-4632-8a51-b0fc88feb9c9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.376274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1357/2710] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:00 2022] PUT /identity/v3/projects/9261f18b1cd84ccbbfb97f028b3318c0/users/87d996f101874598992142bbfd6f3b5c/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:53:00.386941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5de4a-1a04-462b-a89e-15f2ffd08b6b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.387246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5de4a-1a04-462b-a89e-15f2ffd08b6b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.387560 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5de4a-1a04-462b-a89e-15f2ffd08b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.416370 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-20e5de4a-1a04-462b-a89e-15f2ffd08b6b None None] MFA Rules not processed for user `87d996f101874598992142bbfd6f3b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:00.417584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-20e5de4a-1a04-462b-a89e-15f2ffd08b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:00.481864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1354/2711] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:00.486469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cf7d7-6e77-4632-8a51-b0fc88feb9c9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.514111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.553240 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cf7d7-6e77-4632-8a51-b0fc88feb9c9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.558370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09560f94-8cd6-4084-b0d8-427d3909218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.559038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09560f94-8cd6-4084-b0d8-427d3909218f admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.559327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09560f94-8cd6-4084-b0d8-427d3909218f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.559645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09560f94-8cd6-4084-b0d8-427d3909218f admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.560350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09560f94-8cd6-4084-b0d8-427d3909218f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.564348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09560f94-8cd6-4084-b0d8-427d3909218f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.594330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1358/2712] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:00 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:00.598477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5de4a-1a04-462b-a89e-15f2ffd08b6b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.609622 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5de4a-1a04-462b-a89e-15f2ffd08b6b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.672481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67554-6b63-49fe-95ce-9cdab5127fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.673246 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67554-6b63-49fe-95ce-9cdab5127fde admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.673534 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67554-6b63-49fe-95ce-9cdab5127fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.673814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67554-6b63-49fe-95ce-9cdab5127fde admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.674727 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e67554-6b63-49fe-95ce-9cdab5127fde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.677923 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e67554-6b63-49fe-95ce-9cdab5127fde admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.699564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1355/2713] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:00 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:00.703627 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09560f94-8cd6-4084-b0d8-427d3909218f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.715234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09560f94-8cd6-4084-b0d8-427d3909218f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.720128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19379e2-520e-4a27-9187-6cccca0b8b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.720843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19379e2-520e-4a27-9187-6cccca0b8b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.721135 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19379e2-520e-4a27-9187-6cccca0b8b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.721409 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19379e2-520e-4a27-9187-6cccca0b8b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.722169 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19379e2-520e-4a27-9187-6cccca0b8b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.723873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19379e2-520e-4a27-9187-6cccca0b8b1e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.734922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1359/2714] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:53:00.739742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67554-6b63-49fe-95ce-9cdab5127fde admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.750694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67554-6b63-49fe-95ce-9cdab5127fde admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.755498 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2834936-c9ba-4c88-91f7-e27797989b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.756397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2834936-c9ba-4c88-91f7-e27797989b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.756680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2834936-c9ba-4c88-91f7-e27797989b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.756961 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2834936-c9ba-4c88-91f7-e27797989b9e admin admin] PATH_INFO: `/v3/projects/80bf26a5f3df447f974fdf71ef705864/users/5549c0b6d54a46a7bba075150571c5f3/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.772105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2834936-c9ba-4c88-91f7-e27797989b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bf26a5f3df447f974fdf71ef705864, user_id=5549c0b6d54a46a7bba075150571c5f3, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.774795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2834936-c9ba-4c88-91f7-e27797989b9e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.789211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1356/2715] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:00 2022] PUT /identity/v3/projects/80bf26a5f3df447f974fdf71ef705864/users/5549c0b6d54a46a7bba075150571c5f3/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:53:00.793096 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19379e2-520e-4a27-9187-6cccca0b8b1e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.803840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19379e2-520e-4a27-9187-6cccca0b8b1e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.862335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a2363-730c-4d87-9896-e36acbaa0f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.863013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a2363-730c-4d87-9896-e36acbaa0f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.863299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a2363-730c-4d87-9896-e36acbaa0f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.863608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a2363-730c-4d87-9896-e36acbaa0f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.864332 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a2363-730c-4d87-9896-e36acbaa0f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.865943 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a2363-730c-4d87-9896-e36acbaa0f68 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.875808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1360/2716] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:00.881878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2834936-c9ba-4c88-91f7-e27797989b9e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.887882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.895036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2834936-c9ba-4c88-91f7-e27797989b9e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.900788 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da18953-0c49-4785-9ca7-80881149553f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.901685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da18953-0c49-4785-9ca7-80881149553f admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.901971 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da18953-0c49-4785-9ca7-80881149553f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.902255 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da18953-0c49-4785-9ca7-80881149553f admin admin] PATH_INFO: `/v3/projects/80bf26a5f3df447f974fdf71ef705864/users/5549c0b6d54a46a7bba075150571c5f3/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.904348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da18953-0c49-4785-9ca7-80881149553f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bf26a5f3df447f974fdf71ef705864, user_id=5549c0b6d54a46a7bba075150571c5f3, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.907151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da18953-0c49-4785-9ca7-80881149553f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.917632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1357/2717] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:00 2022] PUT /identity/v3/projects/80bf26a5f3df447f974fdf71ef705864/users/5549c0b6d54a46a7bba075150571c5f3/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:00.928819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2a96-12b8-479d-b658-4d1261f936e0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.929142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2a96-12b8-479d-b658-4d1261f936e0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.929428 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2a96-12b8-479d-b658-4d1261f936e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.956709 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1eba2a96-12b8-479d-b658-4d1261f936e0 None None] MFA Rules not processed for user `5549c0b6d54a46a7bba075150571c5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:00.957943 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1eba2a96-12b8-479d-b658-4d1261f936e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:01.028271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1361/2718] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:01.041976 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da18953-0c49-4785-9ca7-80881149553f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:01.043945 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4da18953-0c49-4785-9ca7-80881149553f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:01.107533 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da18953-0c49-4785-9ca7-80881149553f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:01.112001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7c2a-9089-4d05-971e-0e0623ab6c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:01.112655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7c2a-9089-4d05-971e-0e0623ab6c41 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.112936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7c2a-9089-4d05-971e-0e0623ab6c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.113192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7c2a-9089-4d05-971e-0e0623ab6c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.122619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7c2a-9089-4d05-971e-0e0623ab6c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.124093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7c2a-9089-4d05-971e-0e0623ab6c41 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:01.133704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1358/2719] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:01.163233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2a96-12b8-479d-b658-4d1261f936e0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:01.183828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:01.229662 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2a96-12b8-479d-b658-4d1261f936e0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:01.235976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:01.236902 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.237183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.237453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] PATH_INFO: `/v3/users/87d996f101874598992142bbfd6f3b5c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.238697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d996f101874598992142bbfd6f3b5c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.240724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:01.289531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d996f101874598992142bbfd6f3b5c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:01.290216 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d996f101874598992142bbfd6f3b5c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:01.303712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d996f101874598992142bbfd6f3b5c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:01.311533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d996f101874598992142bbfd6f3b5c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:01.320853 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d996f101874598992142bbfd6f3b5c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:01.327076 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1362/2720] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:01 2022] DELETE /identity/v3/users/87d996f101874598992142bbfd6f3b5c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:01.331745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7693f11-d127-444e-99ef-8aaf1b2d89d2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.332043 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7693f11-d127-444e-99ef-8aaf1b2d89d2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.332368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7693f11-d127-444e-99ef-8aaf1b2d89d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.350365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d7693f11-d127-444e-99ef-8aaf1b2d89d2 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:01.351612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d7693f11-d127-444e-99ef-8aaf1b2d89d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:01.413578 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1359/2721] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:01.423905 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:01.425434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:01.484265 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbecbb6-3f7f-4f0c-a4f1-2f1eea46a889 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:01.488831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a02c8-7a54-4a61-8469-ad29c4393d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:01.489594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a02c8-7a54-4a61-8469-ad29c4393d52 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.489991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a02c8-7a54-4a61-8469-ad29c4393d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.490153 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a02c8-7a54-4a61-8469-ad29c4393d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.500317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a02c8-7a54-4a61-8469-ad29c4393d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.501862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a02c8-7a54-4a61-8469-ad29c4393d52 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:01.512137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1363/2722] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:01.890619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7693f11-d127-444e-99ef-8aaf1b2d89d2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:01.892229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d7693f11-d127-444e-99ef-8aaf1b2d89d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:01.905703 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7693f11-d127-444e-99ef-8aaf1b2d89d2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:01.910794 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:01.914431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.914431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.914431 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] PATH_INFO: `/v3/projects/9261f18b1cd84ccbbfb97f028b3318c0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.915286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9261f18b1cd84ccbbfb97f028b3318c0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.917679 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:01.983407 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9261f18b1cd84ccbbfb97f028b3318c0', 'request_id': 'req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:01.991692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9261f18b1cd84ccbbfb97f028b3318c0', 'request_id': 'req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:01.999007 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9261f18b1cd84ccbbfb97f028b3318c0', 'request_id': 'req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.014876 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] The token cache is being invalidate because project 9261f18b1cd84ccbbfb97f028b3318c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:02.015412 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.018285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.020052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1360/2723] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:01 2022] DELETE /identity/v3/projects/9261f18b1cd84ccbbfb97f028b3318c0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:02.024466 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a02c8-7a54-4a61-8469-ad29c4393d52 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:02.035753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a02c8-7a54-4a61-8469-ad29c4393d52 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:02.083534 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:02.108558 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:02.109638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.109993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.111099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] PATH_INFO: `/v3/users/5549c0b6d54a46a7bba075150571c5f3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.112100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5549c0b6d54a46a7bba075150571c5f3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:02.114315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:02.162890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5549c0b6d54a46a7bba075150571c5f3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.163583 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5549c0b6d54a46a7bba075150571c5f3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.176196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5549c0b6d54a46a7bba075150571c5f3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.184142 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5549c0b6d54a46a7bba075150571c5f3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.192310 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-817e27ad-daea-42d1-81ba-4c2ed152d5f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5549c0b6d54a46a7bba075150571c5f3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.198246 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1364/2724] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:02 2022] DELETE /identity/v3/users/5549c0b6d54a46a7bba075150571c5f3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:02.567855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:02.569479 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:02.584891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75eecd-fdc9-4968-a2ac-4aaa4f892fa8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:02.598016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:02.646276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:02.647112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.647371 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.647692 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] PATH_INFO: `/v3/projects/80bf26a5f3df447f974fdf71ef705864` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.648980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=80bf26a5f3df447f974fdf71ef705864)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:02.650900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:02.698329 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80bf26a5f3df447f974fdf71ef705864', 'request_id': 'req-cec8abda-404e-426c-a4e1-d72cc58ad98e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.704776 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80bf26a5f3df447f974fdf71ef705864', 'request_id': 'req-cec8abda-404e-426c-a4e1-d72cc58ad98e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.711697 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80bf26a5f3df447f974fdf71ef705864', 'request_id': 'req-cec8abda-404e-426c-a4e1-d72cc58ad98e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.725153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] The token cache is being invalidate because project 80bf26a5f3df447f974fdf71ef705864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:02.725442 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.726066 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cec8abda-404e-426c-a4e1-d72cc58ad98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:02.727698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1361/2725] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:02 2022] DELETE /identity/v3/projects/80bf26a5f3df447f974fdf71ef705864 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:02.732806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4af50-d720-4822-b898-44eacb4cfd55 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.733172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4af50-d720-4822-b898-44eacb4cfd55 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.733512 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4af50-d720-4822-b898-44eacb4cfd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.751774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-07d4af50-d720-4822-b898-44eacb4cfd55 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:02.753095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-07d4af50-d720-4822-b898-44eacb4cfd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:02.815531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1365/2726] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:02.823699 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234ac78-d434-493e-9d7c-b1da50976362 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.823994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234ac78-d434-493e-9d7c-b1da50976362 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.824277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234ac78-d434-493e-9d7c-b1da50976362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.843813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1234ac78-d434-493e-9d7c-b1da50976362 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:02.845046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1234ac78-d434-493e-9d7c-b1da50976362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:02.854811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1362/2727] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:02.859100 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4af50-d720-4822-b898-44eacb4cfd55 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:02.869064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4af50-d720-4822-b898-44eacb4cfd55 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:02.877967 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a559a61-1aa5-423b-8e10-4181ca9c66c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:02.878646 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a559a61-1aa5-423b-8e10-4181ca9c66c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.878931 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a559a61-1aa5-423b-8e10-4181ca9c66c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.879676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a559a61-1aa5-423b-8e10-4181ca9c66c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.880465 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a559a61-1aa5-423b-8e10-4181ca9c66c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:02.880776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a559a61-1aa5-423b-8e10-4181ca9c66c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:02.882733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a559a61-1aa5-423b-8e10-4181ca9c66c9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:02.894954 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1366/2728] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:02.899586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234ac78-d434-493e-9d7c-b1da50976362 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:02.909362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234ac78-d434-493e-9d7c-b1da50976362 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:02.914256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2a7b7-0c23-465c-a631-a0790219c831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:02.914928 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2a7b7-0c23-465c-a631-a0790219c831 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.915219 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2a7b7-0c23-465c-a631-a0790219c831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.915531 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2a7b7-0c23-465c-a631-a0790219c831 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.916236 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2a7b7-0c23-465c-a631-a0790219c831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:02.918147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2a7b7-0c23-465c-a631-a0790219c831 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:02.946922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1363/2729] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:02 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:02.951508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a559a61-1aa5-423b-8e10-4181ca9c66c9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:02.961459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a559a61-1aa5-423b-8e10-4181ca9c66c9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.005785 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:03.021292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54040-dbbd-4c6b-ba0a-f5e2bbb0b4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.021987 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54040-dbbd-4c6b-ba0a-f5e2bbb0b4dd admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.022276 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54040-dbbd-4c6b-ba0a-f5e2bbb0b4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.023140 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54040-dbbd-4c6b-ba0a-f5e2bbb0b4dd admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.023291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f54040-dbbd-4c6b-ba0a-f5e2bbb0b4dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.025459 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f54040-dbbd-4c6b-ba0a-f5e2bbb0b4dd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.047389 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1367/2730] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:02 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:03.052767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2a7b7-0c23-465c-a631-a0790219c831 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.065131 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2a7b7-0c23-465c-a631-a0790219c831 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.070061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808778a4-0622-440a-adf3-8adaee3d92e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.070760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808778a4-0622-440a-adf3-8adaee3d92e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.071044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808778a4-0622-440a-adf3-8adaee3d92e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.071322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808778a4-0622-440a-adf3-8adaee3d92e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.072133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808778a4-0622-440a-adf3-8adaee3d92e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.073773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808778a4-0622-440a-adf3-8adaee3d92e6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.085331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1364/2731] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:03.089765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54040-dbbd-4c6b-ba0a-f5e2bbb0b4dd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.100057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54040-dbbd-4c6b-ba0a-f5e2bbb0b4dd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.104855 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc5301-005b-4de7-98be-c1040536e5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.105546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc5301-005b-4de7-98be-c1040536e5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.105833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc5301-005b-4de7-98be-c1040536e5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.106113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc5301-005b-4de7-98be-c1040536e5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.106869 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc5301-005b-4de7-98be-c1040536e5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.108539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc5301-005b-4de7-98be-c1040536e5f4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.118887 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1368/2732] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:03.123278 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808778a4-0622-440a-adf3-8adaee3d92e6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.133739 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808778a4-0622-440a-adf3-8adaee3d92e6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.138389 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970de1a-411f-4245-b124-ac1ee9eb1e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.139372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970de1a-411f-4245-b124-ac1ee9eb1e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.139693 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970de1a-411f-4245-b124-ac1ee9eb1e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.139975 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970de1a-411f-4245-b124-ac1ee9eb1e79 admin admin] PATH_INFO: `/v3/projects/62dd014d164a48d49375d1c8fc9a9007/users/d0971d39849b41f68e5724a3a73afbca/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.155495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970de1a-411f-4245-b124-ac1ee9eb1e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62dd014d164a48d49375d1c8fc9a9007, user_id=d0971d39849b41f68e5724a3a73afbca, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.158124 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970de1a-411f-4245-b124-ac1ee9eb1e79 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.172714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1365/2733] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:03 2022] PUT /identity/v3/projects/62dd014d164a48d49375d1c8fc9a9007/users/d0971d39849b41f68e5724a3a73afbca/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:53:03.184425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5c42f-f34f-47e3-8791-033eafeb99b1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.184732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5c42f-f34f-47e3-8791-033eafeb99b1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.185012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5c42f-f34f-47e3-8791-033eafeb99b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.213615 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-23f5c42f-f34f-47e3-8791-033eafeb99b1 None None] MFA Rules not processed for user `d0971d39849b41f68e5724a3a73afbca`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:03.214849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-23f5c42f-f34f-47e3-8791-033eafeb99b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:03.285208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1369/2734] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:03.295575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970de1a-411f-4245-b124-ac1ee9eb1e79 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.297439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1970de1a-411f-4245-b124-ac1ee9eb1e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:03.343914 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:03.356640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970de1a-411f-4245-b124-ac1ee9eb1e79 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.361286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22e3f1-5cf9-46d9-aa8a-1df78b3d3299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.361951 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22e3f1-5cf9-46d9-aa8a-1df78b3d3299 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.362240 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22e3f1-5cf9-46d9-aa8a-1df78b3d3299 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.362524 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22e3f1-5cf9-46d9-aa8a-1df78b3d3299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.376300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22e3f1-5cf9-46d9-aa8a-1df78b3d3299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.377869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22e3f1-5cf9-46d9-aa8a-1df78b3d3299 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.387731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1366/2735] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:13.282095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:13.383634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:14.744341 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70902aa-164c-4f96-88d0-6fc2a949900a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:14.745368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70902aa-164c-4f96-88d0-6fc2a949900a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:14.745912 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70902aa-164c-4f96-88d0-6fc2a949900a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:14.828655 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b70902aa-164c-4f96-88d0-6fc2a949900a None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:14.830125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b70902aa-164c-4f96-88d0-6fc2a949900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:14.923670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1370/2736] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:53:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:14.976301 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22e3f1-5cf9-46d9-aa8a-1df78b3d3299 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:14.977777 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ae22e3f1-5cf9-46d9-aa8a-1df78b3d3299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:15.073514 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22e3f1-5cf9-46d9-aa8a-1df78b3d3299 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:15.081182 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6b36e-f159-4f0c-99a2-f5537a0773db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:15.081844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b36e-f159-4f0c-99a2-f5537a0773db service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:15.082133 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b36e-f159-4f0c-99a2-f5537a0773db service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:15.082415 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b36e-f159-4f0c-99a2-f5537a0773db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:15.091904 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6b36e-f159-4f0c-99a2-f5537a0773db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:15.093506 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6b36e-f159-4f0c-99a2-f5537a0773db service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:15.101797 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1367/2737] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:53:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 20:53:17.166499 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70902aa-164c-4f96-88d0-6fc2a949900a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:17.253425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70902aa-164c-4f96-88d0-6fc2a949900a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:17.260407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:17.261714 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:17.262196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:17.262635 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] PATH_INFO: `/v3/users/d0971d39849b41f68e5724a3a73afbca` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:17.264292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0971d39849b41f68e5724a3a73afbca)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:17.267829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:17.339706 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0971d39849b41f68e5724a3a73afbca'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:17.340415 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0971d39849b41f68e5724a3a73afbca'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:17.355161 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0971d39849b41f68e5724a3a73afbca'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:17.363515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0971d39849b41f68e5724a3a73afbca'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:17.366171 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:17.377790 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0971d39849b41f68e5724a3a73afbca'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:17.389722 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1371/2738] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:17 2022] DELETE /identity/v3/users/d0971d39849b41f68e5724a3a73afbca => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:17.394704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f0a60-8495-4833-b509-c4791f43c6a4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:17.395668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f0a60-8495-4833-b509-c4791f43c6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:17.396172 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f0a60-8495-4833-b509-c4791f43c6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:17.428753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0c9f0a60-8495-4833-b509-c4791f43c6a4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:17.430162 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0c9f0a60-8495-4833-b509-c4791f43c6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:17.515539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1368/2739] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:53:17.528191 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:17.529884 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:17.596040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99319a2d-9f4a-4789-969e-27382009c058 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:17.600515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7581e-a0b1-4703-aec2-3f76cdf487f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:17.601290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7581e-a0b1-4703-aec2-3f76cdf487f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:17.601637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7581e-a0b1-4703-aec2-3f76cdf487f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:17.601984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7581e-a0b1-4703-aec2-3f76cdf487f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:17.611975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7581e-a0b1-4703-aec2-3f76cdf487f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:17.614238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7581e-a0b1-4703-aec2-3f76cdf487f0 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:17.638374 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1372/2740] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:17.943589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f0a60-8495-4833-b509-c4791f43c6a4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:17.945147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0c9f0a60-8495-4833-b509-c4791f43c6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:17.962601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f0a60-8495-4833-b509-c4791f43c6a4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:17.975226 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:17.976247 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:17.976537 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:17.976834 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] PATH_INFO: `/v3/projects/62dd014d164a48d49375d1c8fc9a9007` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:17.978281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62dd014d164a48d49375d1c8fc9a9007)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:17.980293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.042320 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62dd014d164a48d49375d1c8fc9a9007', 'request_id': 'req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:18.049492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62dd014d164a48d49375d1c8fc9a9007', 'request_id': 'req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:18.057214 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62dd014d164a48d49375d1c8fc9a9007', 'request_id': 'req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:18.060146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:18.112978 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] The token cache is being invalidate because project 62dd014d164a48d49375d1c8fc9a9007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:18.113485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:18.114420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f8a7d35-eea8-4276-a72a-3e0c538d7d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:18.117707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1369/2741] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:17 2022] DELETE /identity/v3/projects/62dd014d164a48d49375d1c8fc9a9007 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:18.122505 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b625628-9be2-4ab7-beea-fd787a400d82 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.122898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b625628-9be2-4ab7-beea-fd787a400d82 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.123238 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b625628-9be2-4ab7-beea-fd787a400d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.171317 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5b625628-9be2-4ab7-beea-fd787a400d82 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:18.172684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5b625628-9be2-4ab7-beea-fd787a400d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:18.256290 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f21c38-5d26-42f4-b5d2-2da1862e2a64 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.256761 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f21c38-5d26-42f4-b5d2-2da1862e2a64 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.257106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f21c38-5d26-42f4-b5d2-2da1862e2a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.259820 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1373/2742] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:18.296913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-18f21c38-5d26-42f4-b5d2-2da1862e2a64 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:18.298580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-18f21c38-5d26-42f4-b5d2-2da1862e2a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:18.307249 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1370/2743] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:18.311842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b625628-9be2-4ab7-beea-fd787a400d82 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.325343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b625628-9be2-4ab7-beea-fd787a400d82 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.339624 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f46b9-a715-4d84-92c0-8034ff366eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.340315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f46b9-a715-4d84-92c0-8034ff366eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.340600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f46b9-a715-4d84-92c0-8034ff366eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.340910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f46b9-a715-4d84-92c0-8034ff366eb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.341782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f46b9-a715-4d84-92c0-8034ff366eb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:18.341924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f46b9-a715-4d84-92c0-8034ff366eb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.346398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f46b9-a715-4d84-92c0-8034ff366eb3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.393198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1374/2744] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:18.398160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f21c38-5d26-42f4-b5d2-2da1862e2a64 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.416074 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f21c38-5d26-42f4-b5d2-2da1862e2a64 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.429002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae578-4bb8-4eab-a826-6d764b480e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.429883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae578-4bb8-4eab-a826-6d764b480e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.430351 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae578-4bb8-4eab-a826-6d764b480e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.434337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae578-4bb8-4eab-a826-6d764b480e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.435262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae578-4bb8-4eab-a826-6d764b480e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.441740 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae578-4bb8-4eab-a826-6d764b480e92 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.501094 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1371/2745] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:18 2022] POST /identity/v3/projects => generated 361 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:18.505301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f46b9-a715-4d84-92c0-8034ff366eb3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.521756 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f46b9-a715-4d84-92c0-8034ff366eb3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.569128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae578-4bb8-4eab-a826-6d764b480e92 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.571414 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-09eae578-4bb8-4eab-a826-6d764b480e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:18.590828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c3da9-eca0-4494-bbbc-e8fe313020be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.591644 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c3da9-eca0-4494-bbbc-e8fe313020be admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.591930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c3da9-eca0-4494-bbbc-e8fe313020be admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.592334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c3da9-eca0-4494-bbbc-e8fe313020be admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.593065 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c3da9-eca0-4494-bbbc-e8fe313020be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.595210 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c3da9-eca0-4494-bbbc-e8fe313020be admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.621291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1375/2746] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:18 2022] POST /identity/v3/users => generated 281 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:18.625448 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c3da9-eca0-4494-bbbc-e8fe313020be admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.637582 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c3da9-eca0-4494-bbbc-e8fe313020be admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.642479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2139f-4f07-4750-9e80-c043f382b940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.643798 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2139f-4f07-4750-9e80-c043f382b940 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.644084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2139f-4f07-4750-9e80-c043f382b940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.644363 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2139f-4f07-4750-9e80-c043f382b940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.645102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2139f-4f07-4750-9e80-c043f382b940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.646732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2139f-4f07-4750-9e80-c043f382b940 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.653633 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:18.661575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1376/2747] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:53:18.667196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2139f-4f07-4750-9e80-c043f382b940 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.680319 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2139f-4f07-4750-9e80-c043f382b940 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.680626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae578-4bb8-4eab-a826-6d764b480e92 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.685095 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeef42-6784-450d-bec9-b21037dc09ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.685857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeef42-6784-450d-bec9-b21037dc09ed service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.686143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeef42-6784-450d-bec9-b21037dc09ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.686523 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeef42-6784-450d-bec9-b21037dc09ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.689117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c7eeef42-6784-450d-bec9-b21037dc09ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:18.701533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c7bf0-1249-4e33-af14-8e968304e4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.702277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c7bf0-1249-4e33-af14-8e968304e4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.702591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c7bf0-1249-4e33-af14-8e968304e4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.702898 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c7bf0-1249-4e33-af14-8e968304e4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.703677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c7bf0-1249-4e33-af14-8e968304e4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.708219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c7bf0-1249-4e33-af14-8e968304e4b8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.731837 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1377/2748] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:18.732823 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c7bf0-1249-4e33-af14-8e968304e4b8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.745166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c7bf0-1249-4e33-af14-8e968304e4b8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.751822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc39bb-2b71-485a-90d2-34fec144e9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.752982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc39bb-2b71-485a-90d2-34fec144e9bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.753272 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc39bb-2b71-485a-90d2-34fec144e9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.753574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc39bb-2b71-485a-90d2-34fec144e9bd admin admin] PATH_INFO: `/v3/projects/8392dae07f184506b06a3022b6e43ff1/users/5ff601b676a6415bb466d679cef444a5/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.771628 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeef42-6784-450d-bec9-b21037dc09ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.773316 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeef42-6784-450d-bec9-b21037dc09ed service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.778443 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc39bb-2b71-485a-90d2-34fec144e9bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8392dae07f184506b06a3022b6e43ff1, user_id=5ff601b676a6415bb466d679cef444a5, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.781847 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc39bb-2b71-485a-90d2-34fec144e9bd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.787024 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1372/2749] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:53:18.804917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1378/2750] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:18 2022] PUT /identity/v3/projects/8392dae07f184506b06a3022b6e43ff1/users/5ff601b676a6415bb466d679cef444a5/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:18.821753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1886942-eee1-4c7e-a751-fb7c1aa921ac None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.822281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1886942-eee1-4c7e-a751-fb7c1aa921ac None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.822738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1886942-eee1-4c7e-a751-fb7c1aa921ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.868545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a1886942-eee1-4c7e-a751-fb7c1aa921ac None None] MFA Rules not processed for user `5ff601b676a6415bb466d679cef444a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:18.870035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a1886942-eee1-4c7e-a751-fb7c1aa921ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:18.958590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1373/2751] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:18.998101 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc39bb-2b71-485a-90d2-34fec144e9bd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:19.000064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5afc39bb-2b71-485a-90d2-34fec144e9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:19.073368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc39bb-2b71-485a-90d2-34fec144e9bd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:19.077493 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e40be4-7379-447a-8066-a022bb363cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:19.078275 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e40be4-7379-447a-8066-a022bb363cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:19.078656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e40be4-7379-447a-8066-a022bb363cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:19.079027 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e40be4-7379-447a-8066-a022bb363cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:19.089326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e40be4-7379-447a-8066-a022bb363cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:19.091043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e40be4-7379-447a-8066-a022bb363cf6 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:19.102958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1379/2752] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:53:19.119328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1886942-eee1-4c7e-a751-fb7c1aa921ac None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:19.128033 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:19.237613 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1886942-eee1-4c7e-a751-fb7c1aa921ac None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:19.251902 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:19.253503 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:19.253885 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:19.255933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] PATH_INFO: `/v3/users/5ff601b676a6415bb466d679cef444a5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:19.259119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff601b676a6415bb466d679cef444a5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:19.261277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:19.354500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff601b676a6415bb466d679cef444a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:19.355333 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff601b676a6415bb466d679cef444a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:19.372440 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff601b676a6415bb466d679cef444a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:19.380106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff601b676a6415bb466d679cef444a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:19.380981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff601b676a6415bb466d679cef444a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:19.404644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1374/2753] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:19 2022] DELETE /identity/v3/users/5ff601b676a6415bb466d679cef444a5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:19.409623 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d04fc-866b-435f-9ac8-4aa89957f01b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:19.410617 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d04fc-866b-435f-9ac8-4aa89957f01b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:19.410856 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d04fc-866b-435f-9ac8-4aa89957f01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:19.441299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-051d04fc-866b-435f-9ac8-4aa89957f01b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:19.442605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-051d04fc-866b-435f-9ac8-4aa89957f01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:19.534651 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1380/2754] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:53:19.545763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:19.547642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:19.635099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c30f8e-66ba-499f-9685-7158e8e02742 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:19.642824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21274425-ea16-476f-8627-f112b42d00d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:19.644421 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21274425-ea16-476f-8627-f112b42d00d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:19.649058 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21274425-ea16-476f-8627-f112b42d00d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:19.649981 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21274425-ea16-476f-8627-f112b42d00d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:19.677653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21274425-ea16-476f-8627-f112b42d00d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:19.683148 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21274425-ea16-476f-8627-f112b42d00d6 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:19.699745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1375/2755] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:20.054801 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d04fc-866b-435f-9ac8-4aa89957f01b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.057859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-051d04fc-866b-435f-9ac8-4aa89957f01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:20.081657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d04fc-866b-435f-9ac8-4aa89957f01b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.090607 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.091747 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.092244 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.092832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] PATH_INFO: `/v3/projects/8392dae07f184506b06a3022b6e43ff1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.094479 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] RBAC: Authorizing `identity:delete_project(project_id=8392dae07f184506b06a3022b6e43ff1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.096961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:20.172727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8392dae07f184506b06a3022b6e43ff1', 'request_id': 'req-9944f06c-85e2-4218-a38f-09b9b5795eab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:20.183258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8392dae07f184506b06a3022b6e43ff1', 'request_id': 'req-9944f06c-85e2-4218-a38f-09b9b5795eab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:20.190425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:20.192173 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8392dae07f184506b06a3022b6e43ff1', 'request_id': 'req-9944f06c-85e2-4218-a38f-09b9b5795eab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:20.211154 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] The token cache is being invalidate because project 8392dae07f184506b06a3022b6e43ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:20.211639 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:20.214738 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9944f06c-85e2-4218-a38f-09b9b5795eab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:20.217012 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1381/2756] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:20 2022] DELETE /identity/v3/projects/8392dae07f184506b06a3022b6e43ff1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:20.222388 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592301d-47b0-40a3-add1-bc8af65b86f8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.223063 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592301d-47b0-40a3-add1-bc8af65b86f8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.223434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592301d-47b0-40a3-add1-bc8af65b86f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.249953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f592301d-47b0-40a3-add1-bc8af65b86f8 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:20.251305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f592301d-47b0-40a3-add1-bc8af65b86f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:20.283243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:20.350173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1376/2757] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:20.357128 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8f1b3-3aac-4ec8-90ee-1c86d53cfd2d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.357669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8f1b3-3aac-4ec8-90ee-1c86d53cfd2d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.357960 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8f1b3-3aac-4ec8-90ee-1c86d53cfd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.391162 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-43c8f1b3-3aac-4ec8-90ee-1c86d53cfd2d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:20.392444 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-43c8f1b3-3aac-4ec8-90ee-1c86d53cfd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:20.406751 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1382/2758] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:20.411060 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592301d-47b0-40a3-add1-bc8af65b86f8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.426256 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592301d-47b0-40a3-add1-bc8af65b86f8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.431434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a7091-3391-420c-a952-efb0b9377dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.432145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a7091-3391-420c-a952-efb0b9377dee admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.432425 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a7091-3391-420c-a952-efb0b9377dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.432710 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a7091-3391-420c-a952-efb0b9377dee admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.433437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a7091-3391-420c-a952-efb0b9377dee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:20.433743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a7091-3391-420c-a952-efb0b9377dee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.435340 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a7091-3391-420c-a952-efb0b9377dee admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:20.448430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1377/2759] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:20.453040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8f1b3-3aac-4ec8-90ee-1c86d53cfd2d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.463070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8f1b3-3aac-4ec8-90ee-1c86d53cfd2d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.471993 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edc0e-f90a-469d-b5f6-e7ceffcb514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.472677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edc0e-f90a-469d-b5f6-e7ceffcb514c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.472970 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edc0e-f90a-469d-b5f6-e7ceffcb514c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.473254 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edc0e-f90a-469d-b5f6-e7ceffcb514c admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.473971 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edc0e-f90a-469d-b5f6-e7ceffcb514c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.475958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edc0e-f90a-469d-b5f6-e7ceffcb514c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:20.519728 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1383/2760] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:20 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:20.523711 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a7091-3391-420c-a952-efb0b9377dee admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.534941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a7091-3391-420c-a952-efb0b9377dee admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.619989 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b47f8-5a8b-40b0-a091-26cfa1a980a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.620592 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b47f8-5a8b-40b0-a091-26cfa1a980a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.620878 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b47f8-5a8b-40b0-a091-26cfa1a980a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.621158 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b47f8-5a8b-40b0-a091-26cfa1a980a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.621871 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b47f8-5a8b-40b0-a091-26cfa1a980a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.625550 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b47f8-5a8b-40b0-a091-26cfa1a980a4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:20.648337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1378/2761] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:20 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:20.653050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edc0e-f90a-469d-b5f6-e7ceffcb514c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.664581 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edc0e-f90a-469d-b5f6-e7ceffcb514c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.669429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3fa2d-65f9-4f96-b9e5-48bb4c711205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.670195 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3fa2d-65f9-4f96-b9e5-48bb4c711205 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.670483 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3fa2d-65f9-4f96-b9e5-48bb4c711205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.670964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3fa2d-65f9-4f96-b9e5-48bb4c711205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.671776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3fa2d-65f9-4f96-b9e5-48bb4c711205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.673514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3fa2d-65f9-4f96-b9e5-48bb4c711205 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:20.686696 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1384/2762] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:20.691315 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b47f8-5a8b-40b0-a091-26cfa1a980a4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.703901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b47f8-5a8b-40b0-a091-26cfa1a980a4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.712295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4873002-3205-4878-a8a3-30f441143fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.713368 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4873002-3205-4878-a8a3-30f441143fae admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.713812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4873002-3205-4878-a8a3-30f441143fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.714274 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4873002-3205-4878-a8a3-30f441143fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.715195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4873002-3205-4878-a8a3-30f441143fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.717035 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4873002-3205-4878-a8a3-30f441143fae admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:20.735680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1379/2763] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:20.740456 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3fa2d-65f9-4f96-b9e5-48bb4c711205 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.753354 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3fa2d-65f9-4f96-b9e5-48bb4c711205 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.758503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447caecc-73bb-4398-b04d-2b0f2b77c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.759788 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447caecc-73bb-4398-b04d-2b0f2b77c5eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.760137 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447caecc-73bb-4398-b04d-2b0f2b77c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.760606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447caecc-73bb-4398-b04d-2b0f2b77c5eb admin admin] PATH_INFO: `/v3/projects/6a690889825843b2b7db3f87806d9cfd/users/79db66d78174491cbc08384153711601/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.780020 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447caecc-73bb-4398-b04d-2b0f2b77c5eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a690889825843b2b7db3f87806d9cfd, user_id=79db66d78174491cbc08384153711601, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.782780 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447caecc-73bb-4398-b04d-2b0f2b77c5eb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:20.802719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1385/2764] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:20 2022] PUT /identity/v3/projects/6a690889825843b2b7db3f87806d9cfd/users/79db66d78174491cbc08384153711601/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:20.815280 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290728d-5ff4-499c-9648-36dcb91922a0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.815948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290728d-5ff4-499c-9648-36dcb91922a0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.816296 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290728d-5ff4-499c-9648-36dcb91922a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.867145 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3290728d-5ff4-499c-9648-36dcb91922a0 None None] MFA Rules not processed for user `79db66d78174491cbc08384153711601`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:20.869809 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3290728d-5ff4-499c-9648-36dcb91922a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:20.998258 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447caecc-73bb-4398-b04d-2b0f2b77c5eb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.999883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-447caecc-73bb-4398-b04d-2b0f2b77c5eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:21.070219 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1380/2765] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:53:21.082167 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290728d-5ff4-499c-9648-36dcb91922a0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:21.083706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3290728d-5ff4-499c-9648-36dcb91922a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:21.148147 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:21.174105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447caecc-73bb-4398-b04d-2b0f2b77c5eb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:21.190518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e851e5-aab3-448d-83f0-82e47923d20e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:21.191234 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e851e5-aab3-448d-83f0-82e47923d20e service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:21.191638 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e851e5-aab3-448d-83f0-82e47923d20e service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:21.191929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e851e5-aab3-448d-83f0-82e47923d20e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:21.197988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b6e851e5-aab3-448d-83f0-82e47923d20e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:21.221492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290728d-5ff4-499c-9648-36dcb91922a0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:21.227433 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e851e5-aab3-448d-83f0-82e47923d20e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:21.231191 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322203b-d7f7-4609-ad31-f1bb97c4a08d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:21.232139 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322203b-d7f7-4609-ad31-f1bb97c4a08d service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:21.232736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322203b-d7f7-4609-ad31-f1bb97c4a08d service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:21.233026 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e851e5-aab3-448d-83f0-82e47923d20e service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:21.236949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322203b-d7f7-4609-ad31-f1bb97c4a08d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:21.261104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322203b-d7f7-4609-ad31-f1bb97c4a08d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:21.262990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322203b-d7f7-4609-ad31-f1bb97c4a08d service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:21.272322 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1386/2766] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:53:21.298599 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1381/2767] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:53:28.140296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be40569-64a4-4ab0-9f1a-00fe01a6e1c6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:28.140828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be40569-64a4-4ab0-9f1a-00fe01a6e1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:28.140995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be40569-64a4-4ab0-9f1a-00fe01a6e1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:28.160661 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9be40569-64a4-4ab0-9f1a-00fe01a6e1c6 None None] MFA Rules not processed for user `79db66d78174491cbc08384153711601`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:28.161907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9be40569-64a4-4ab0-9f1a-00fe01a6e1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:28.169179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1387/2768] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:31.277824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=16 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:34.768590 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322203b-d7f7-4609-ad31-f1bb97c4a08d service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:34.810481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322203b-d7f7-4609-ad31-f1bb97c4a08d service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:34.816669 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:34.817393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:34.817606 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:34.817883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:34.823269 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:34.846844 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:34.848701 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:34.876029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1382/2769] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:36.345938 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015f089-6a8e-4948-9585-d3912c16a273 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:36.346406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015f089-6a8e-4948-9585-d3912c16a273 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:36.346500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015f089-6a8e-4948-9585-d3912c16a273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:36.364282 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7015f089-6a8e-4948-9585-d3912c16a273 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:36.365474 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7015f089-6a8e-4948-9585-d3912c16a273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:36.421669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1388/2770] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 13 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:36.434332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:36.436019 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:36.518933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5873b53-5560-4723-9f49-b37e3a22d571 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:36.526521 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d52c20-3f86-4e9b-85ba-150761d57986 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:36.527217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d52c20-3f86-4e9b-85ba-150761d57986 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:36.527541 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d52c20-3f86-4e9b-85ba-150761d57986 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:36.527820 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d52c20-3f86-4e9b-85ba-150761d57986 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:36.536769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d52c20-3f86-4e9b-85ba-150761d57986 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:36.538381 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d52c20-3f86-4e9b-85ba-150761d57986 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:36.547166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1383/2771] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:53:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:53:38.569777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015f089-6a8e-4948-9585-d3912c16a273 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:38.571974 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7015f089-6a8e-4948-9585-d3912c16a273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:38.596485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:38.652929 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015f089-6a8e-4948-9585-d3912c16a273 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:38.657572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:38.658478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:38.658760 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:38.659040 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] PATH_INFO: `/v3/users/7cd67f4fdca84f328a7fe02623304f1c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:38.660843 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cd67f4fdca84f328a7fe02623304f1c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:38.662783 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:38.716072 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cd67f4fdca84f328a7fe02623304f1c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:38.716737 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cd67f4fdca84f328a7fe02623304f1c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:38.732061 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cd67f4fdca84f328a7fe02623304f1c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:38.740727 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cd67f4fdca84f328a7fe02623304f1c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:38.748868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-24e82e31-d29f-47e3-8d23-ea854cb07bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cd67f4fdca84f328a7fe02623304f1c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:38.755886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1389/2772] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:38 2022] DELETE /identity/v3/users/7cd67f4fdca84f328a7fe02623304f1c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:38.960270 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d52c20-3f86-4e9b-85ba-150761d57986 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:38.971569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d52c20-3f86-4e9b-85ba-150761d57986 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:39.027978 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:39.028886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.029171 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.029451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] PATH_INFO: `/v3/projects/f840d73c94fa4abc9552d50b0f78fd89` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.030674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f840d73c94fa4abc9552d50b0f78fd89)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:39.032632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:39.081717 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f840d73c94fa4abc9552d50b0f78fd89', 'request_id': 'req-0c1bc40b-5a90-4b34-8275-b77c8c817df0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:39.088773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f840d73c94fa4abc9552d50b0f78fd89', 'request_id': 'req-0c1bc40b-5a90-4b34-8275-b77c8c817df0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:39.100940 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f840d73c94fa4abc9552d50b0f78fd89', 'request_id': 'req-0c1bc40b-5a90-4b34-8275-b77c8c817df0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:39.117798 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] The token cache is being invalidate because project f840d73c94fa4abc9552d50b0f78fd89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:39.118741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:39.119408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0c1bc40b-5a90-4b34-8275-b77c8c817df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:39.121234 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1384/2773] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:38 2022] DELETE /identity/v3/projects/f840d73c94fa4abc9552d50b0f78fd89 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:39.126387 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a277fd-8f42-4dd1-928b-722a9e1d33db None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.126676 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a277fd-8f42-4dd1-928b-722a9e1d33db None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.126978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a277fd-8f42-4dd1-928b-722a9e1d33db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.145752 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e7a277fd-8f42-4dd1-928b-722a9e1d33db None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:39.146991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e7a277fd-8f42-4dd1-928b-722a9e1d33db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:39.207849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1390/2774] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:39.216497 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbdaba-9d73-4c22-96f0-25e473e03d1d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.217406 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbdaba-9d73-4c22-96f0-25e473e03d1d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.217851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbdaba-9d73-4c22-96f0-25e473e03d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.234879 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:39.240675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3efbdaba-9d73-4c22-96f0-25e473e03d1d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:39.242634 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3efbdaba-9d73-4c22-96f0-25e473e03d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:39.251226 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1385/2775] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:39.255732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a277fd-8f42-4dd1-928b-722a9e1d33db None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:39.265671 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a277fd-8f42-4dd1-928b-722a9e1d33db None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:39.270426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c300eb3-7e4d-4a81-9f79-b6e76bfcfe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:39.271091 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300eb3-7e4d-4a81-9f79-b6e76bfcfe1b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.271381 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300eb3-7e4d-4a81-9f79-b6e76bfcfe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.271849 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300eb3-7e4d-4a81-9f79-b6e76bfcfe1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.272604 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c300eb3-7e4d-4a81-9f79-b6e76bfcfe1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:39.272916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c300eb3-7e4d-4a81-9f79-b6e76bfcfe1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:39.274537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c300eb3-7e4d-4a81-9f79-b6e76bfcfe1b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:39.286557 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1391/2776] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:39.291243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbdaba-9d73-4c22-96f0-25e473e03d1d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:39.300837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbdaba-9d73-4c22-96f0-25e473e03d1d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:39.306222 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f1dec-a972-4279-a4bc-9ed735c46593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:39.306894 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f1dec-a972-4279-a4bc-9ed735c46593 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.307184 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f1dec-a972-4279-a4bc-9ed735c46593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.307482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f1dec-a972-4279-a4bc-9ed735c46593 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.308322 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f1dec-a972-4279-a4bc-9ed735c46593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:39.310314 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f1dec-a972-4279-a4bc-9ed735c46593 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:39.342827 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1386/2777] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:39 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:39.344296 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c300eb3-7e4d-4a81-9f79-b6e76bfcfe1b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:39.361038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c300eb3-7e4d-4a81-9f79-b6e76bfcfe1b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:39.380038 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:39.430033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088717-e13f-4573-8e88-96f4ae9ff725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:39.430720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088717-e13f-4573-8e88-96f4ae9ff725 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.431009 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088717-e13f-4573-8e88-96f4ae9ff725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.431287 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088717-e13f-4573-8e88-96f4ae9ff725 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.432008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92088717-e13f-4573-8e88-96f4ae9ff725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:39.433953 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92088717-e13f-4573-8e88-96f4ae9ff725 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:39.453677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1392/2778] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:39 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:39.458283 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f1dec-a972-4279-a4bc-9ed735c46593 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:39.470364 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f1dec-a972-4279-a4bc-9ed735c46593 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:39.475877 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50ffb8-cad3-401c-98fd-2a26fc73e9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:39.476616 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50ffb8-cad3-401c-98fd-2a26fc73e9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.477126 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50ffb8-cad3-401c-98fd-2a26fc73e9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.477644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50ffb8-cad3-401c-98fd-2a26fc73e9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.479001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50ffb8-cad3-401c-98fd-2a26fc73e9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:39.481093 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50ffb8-cad3-401c-98fd-2a26fc73e9ae admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:39.492821 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1387/2779] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:53:39.497324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088717-e13f-4573-8e88-96f4ae9ff725 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:39.508550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088717-e13f-4573-8e88-96f4ae9ff725 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:39.514789 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0430d-6080-442f-9b50-8c2c4450533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:39.515486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0430d-6080-442f-9b50-8c2c4450533d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.515806 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0430d-6080-442f-9b50-8c2c4450533d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.516095 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0430d-6080-442f-9b50-8c2c4450533d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.516824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0430d-6080-442f-9b50-8c2c4450533d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:39.518452 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0430d-6080-442f-9b50-8c2c4450533d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:39.529532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1393/2780] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:39.533839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50ffb8-cad3-401c-98fd-2a26fc73e9ae admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:39.544505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50ffb8-cad3-401c-98fd-2a26fc73e9ae admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:39.550230 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80772f55-46aa-414a-9164-0456fb1d351f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:39.551160 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80772f55-46aa-414a-9164-0456fb1d351f admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.551444 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80772f55-46aa-414a-9164-0456fb1d351f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.551753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80772f55-46aa-414a-9164-0456fb1d351f admin admin] PATH_INFO: `/v3/projects/74db0ff8b54e4926935267b1ced4667a/users/ab077ee5595b44f7b04a70f3c10b3949/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.568040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80772f55-46aa-414a-9164-0456fb1d351f admin admin] RBAC: Authorizing `identity:create_grant(project_id=74db0ff8b54e4926935267b1ced4667a, user_id=ab077ee5595b44f7b04a70f3c10b3949, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:39.570720 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80772f55-46aa-414a-9164-0456fb1d351f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:39.585865 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1388/2781] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:39 2022] PUT /identity/v3/projects/74db0ff8b54e4926935267b1ced4667a/users/ab077ee5595b44f7b04a70f3c10b3949/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:39.596190 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2c007-5635-47bc-8a25-122f61ca9139 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.596719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2c007-5635-47bc-8a25-122f61ca9139 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.596719 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2c007-5635-47bc-8a25-122f61ca9139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.624529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f7b2c007-5635-47bc-8a25-122f61ca9139 None None] MFA Rules not processed for user `ab077ee5595b44f7b04a70f3c10b3949`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:39.625831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f7b2c007-5635-47bc-8a25-122f61ca9139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:39.693274 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1394/2782] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:39.705922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80772f55-46aa-414a-9164-0456fb1d351f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:39.707754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-80772f55-46aa-414a-9164-0456fb1d351f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:39.767922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80772f55-46aa-414a-9164-0456fb1d351f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:39.771655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a279f1-f3f1-47eb-ab64-3c880f5466cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:39.772307 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a279f1-f3f1-47eb-ab64-3c880f5466cd service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.772593 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a279f1-f3f1-47eb-ab64-3c880f5466cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.772869 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a279f1-f3f1-47eb-ab64-3c880f5466cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.781327 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a279f1-f3f1-47eb-ab64-3c880f5466cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:39.782756 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a279f1-f3f1-47eb-ab64-3c880f5466cd service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:39.791477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1389/2783] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:45.520566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2c007-5635-47bc-8a25-122f61ca9139 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:45.593113 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2c007-5635-47bc-8a25-122f61ca9139 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:45.598885 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:45.600002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:45.600521 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:45.600819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] PATH_INFO: `/v3/users/ab077ee5595b44f7b04a70f3c10b3949` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:45.602055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab077ee5595b44f7b04a70f3c10b3949)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:45.605177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:45.644222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:45.669866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab077ee5595b44f7b04a70f3c10b3949'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:45.670616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab077ee5595b44f7b04a70f3c10b3949'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:45.682669 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab077ee5595b44f7b04a70f3c10b3949'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:45.689446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab077ee5595b44f7b04a70f3c10b3949'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:45.697658 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab077ee5595b44f7b04a70f3c10b3949'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:45.704243 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1395/2784] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:45 2022] DELETE /identity/v3/users/ab077ee5595b44f7b04a70f3c10b3949 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:45.709337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3a206-3b76-4b6a-8e28-a6f31845e735 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:45.709842 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3a206-3b76-4b6a-8e28-a6f31845e735 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:45.709941 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3a206-3b76-4b6a-8e28-a6f31845e735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:45.730607 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-30d3a206-3b76-4b6a-8e28-a6f31845e735 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:45.731879 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-30d3a206-3b76-4b6a-8e28-a6f31845e735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:45.794627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1390/2785] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:53:45.805401 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:45.806982 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:45.883189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b105c2b-510d-4947-bfde-f7b931ea51fc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:45.886904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d8011-0ef9-4e96-9db6-ecb5b921af60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:45.887616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d8011-0ef9-4e96-9db6-ecb5b921af60 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:45.887912 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d8011-0ef9-4e96-9db6-ecb5b921af60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:45.888187 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d8011-0ef9-4e96-9db6-ecb5b921af60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:45.898189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d8011-0ef9-4e96-9db6-ecb5b921af60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:45.899712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d8011-0ef9-4e96-9db6-ecb5b921af60 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:45.909549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1396/2786] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:46.189886 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3a206-3b76-4b6a-8e28-a6f31845e735 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:46.191762 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-30d3a206-3b76-4b6a-8e28-a6f31845e735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:46.206906 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3a206-3b76-4b6a-8e28-a6f31845e735 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:46.212564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:46.213463 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.213742 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.214020 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] PATH_INFO: `/v3/projects/74db0ff8b54e4926935267b1ced4667a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.215211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74db0ff8b54e4926935267b1ced4667a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:46.217231 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:46.261111 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:46.269174 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74db0ff8b54e4926935267b1ced4667a', 'request_id': 'req-a00ec5c9-2af7-49e8-831d-c83ca02bf474'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:46.276723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74db0ff8b54e4926935267b1ced4667a', 'request_id': 'req-a00ec5c9-2af7-49e8-831d-c83ca02bf474'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:46.282206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74db0ff8b54e4926935267b1ced4667a', 'request_id': 'req-a00ec5c9-2af7-49e8-831d-c83ca02bf474'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:46.294926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] The token cache is being invalidate because project 74db0ff8b54e4926935267b1ced4667a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:46.295212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:46.295851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a00ec5c9-2af7-49e8-831d-c83ca02bf474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:46.297407 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1391/2787] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:46 2022] DELETE /identity/v3/projects/74db0ff8b54e4926935267b1ced4667a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:53:46.302620 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208a591-df0d-4aa5-ae7a-1d9d59934aae None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.302917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208a591-df0d-4aa5-ae7a-1d9d59934aae None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.303201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208a591-df0d-4aa5-ae7a-1d9d59934aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.324881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5208a591-df0d-4aa5-ae7a-1d9d59934aae None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:46.326131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5208a591-df0d-4aa5-ae7a-1d9d59934aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:46.392976 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1397/2788] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:53:46.401057 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6c780-6e2a-4b1a-862d-45e522fc6e69 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.401346 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6c780-6e2a-4b1a-862d-45e522fc6e69 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.401627 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6c780-6e2a-4b1a-862d-45e522fc6e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.420917 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ace6c780-6e2a-4b1a-862d-45e522fc6e69 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:46.422129 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ace6c780-6e2a-4b1a-862d-45e522fc6e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:46.432012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1392/2789] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:46.436292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208a591-df0d-4aa5-ae7a-1d9d59934aae None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:46.447290 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208a591-df0d-4aa5-ae7a-1d9d59934aae None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:46.452956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3870b9a-4858-4b23-9e15-67a0fd4e5831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:46.453636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3870b9a-4858-4b23-9e15-67a0fd4e5831 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.453919 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3870b9a-4858-4b23-9e15-67a0fd4e5831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.454207 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3870b9a-4858-4b23-9e15-67a0fd4e5831 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.454937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3870b9a-4858-4b23-9e15-67a0fd4e5831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:46.455350 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3870b9a-4858-4b23-9e15-67a0fd4e5831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:46.457179 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3870b9a-4858-4b23-9e15-67a0fd4e5831 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:46.469546 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1398/2790] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:46.474097 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6c780-6e2a-4b1a-862d-45e522fc6e69 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:46.483829 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6c780-6e2a-4b1a-862d-45e522fc6e69 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:46.489185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c569685-ce9e-49bb-aaf2-9be04be579ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:46.489868 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c569685-ce9e-49bb-aaf2-9be04be579ce admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.490151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c569685-ce9e-49bb-aaf2-9be04be579ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.490434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c569685-ce9e-49bb-aaf2-9be04be579ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.491153 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c569685-ce9e-49bb-aaf2-9be04be579ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:46.493146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c569685-ce9e-49bb-aaf2-9be04be579ce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:46.520992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1393/2791] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:46 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:46.526364 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3870b9a-4858-4b23-9e15-67a0fd4e5831 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:46.537753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3870b9a-4858-4b23-9e15-67a0fd4e5831 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:46.572870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:46.599043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671de4a-8830-46ac-afa0-ec2eeb7dc34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:46.599740 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671de4a-8830-46ac-afa0-ec2eeb7dc34c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.600024 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671de4a-8830-46ac-afa0-ec2eeb7dc34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.600300 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671de4a-8830-46ac-afa0-ec2eeb7dc34c admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.600975 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671de4a-8830-46ac-afa0-ec2eeb7dc34c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:46.602889 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671de4a-8830-46ac-afa0-ec2eeb7dc34c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:46.624168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1399/2792] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:46 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:46.628430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c569685-ce9e-49bb-aaf2-9be04be579ce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:46.639418 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c569685-ce9e-49bb-aaf2-9be04be579ce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:46.644781 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd720e-d255-4aa4-b8b5-338910ec1db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:46.645387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd720e-d255-4aa4-b8b5-338910ec1db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.645675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd720e-d255-4aa4-b8b5-338910ec1db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.645957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd720e-d255-4aa4-b8b5-338910ec1db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.646682 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd720e-d255-4aa4-b8b5-338910ec1db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:46.648318 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd720e-d255-4aa4-b8b5-338910ec1db1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:46.659193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1394/2793] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:46.663222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671de4a-8830-46ac-afa0-ec2eeb7dc34c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:46.674264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671de4a-8830-46ac-afa0-ec2eeb7dc34c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:46.680121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e685332-ffb1-4f9b-a390-491c0aad050f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:46.680778 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e685332-ffb1-4f9b-a390-491c0aad050f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.681064 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e685332-ffb1-4f9b-a390-491c0aad050f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.681767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e685332-ffb1-4f9b-a390-491c0aad050f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.682043 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e685332-ffb1-4f9b-a390-491c0aad050f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:46.683792 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e685332-ffb1-4f9b-a390-491c0aad050f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:46.696062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1400/2794] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:46.700420 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd720e-d255-4aa4-b8b5-338910ec1db1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:46.710843 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd720e-d255-4aa4-b8b5-338910ec1db1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:46.718984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308c5c5-b3fd-48d4-9e43-90327becc059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:46.719942 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308c5c5-b3fd-48d4-9e43-90327becc059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.720233 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308c5c5-b3fd-48d4-9e43-90327becc059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.720501 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308c5c5-b3fd-48d4-9e43-90327becc059 admin admin] PATH_INFO: `/v3/projects/c26f9b9ea47148708610563386fcecaa/users/1578d60cec6f435ab9680b88deff76ec/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.736039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308c5c5-b3fd-48d4-9e43-90327becc059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26f9b9ea47148708610563386fcecaa, user_id=1578d60cec6f435ab9680b88deff76ec, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:46.739146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308c5c5-b3fd-48d4-9e43-90327becc059 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:46.761611 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1395/2795] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:46 2022] PUT /identity/v3/projects/c26f9b9ea47148708610563386fcecaa/users/1578d60cec6f435ab9680b88deff76ec/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:46.766945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33d495-7bf1-4bc4-a4a4-257610502542 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.767324 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33d495-7bf1-4bc4-a4a4-257610502542 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.767776 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33d495-7bf1-4bc4-a4a4-257610502542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.787707 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ab33d495-7bf1-4bc4-a4a4-257610502542 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:46.789032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab33d495-7bf1-4bc4-a4a4-257610502542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:46.853528 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1401/2796] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:46.863925 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308c5c5-b3fd-48d4-9e43-90327becc059 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:46.868286 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0308c5c5-b3fd-48d4-9e43-90327becc059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:46.928193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308c5c5-b3fd-48d4-9e43-90327becc059 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:46.933896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333f174-331d-4201-a38b-31254d2117ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:46.934547 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333f174-331d-4201-a38b-31254d2117ad service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:46.934837 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333f174-331d-4201-a38b-31254d2117ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:46.935118 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333f174-331d-4201-a38b-31254d2117ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:46.948309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333f174-331d-4201-a38b-31254d2117ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:46.949813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333f174-331d-4201-a38b-31254d2117ad service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:46.961334 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1396/2797] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:47.075151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33d495-7bf1-4bc4-a4a4-257610502542 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:47.076777 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab33d495-7bf1-4bc4-a4a4-257610502542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:47.089292 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33d495-7bf1-4bc4-a4a4-257610502542 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:47.094291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:47.095312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:47.095725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:47.096165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] PATH_INFO: `/v3/users/da1bff9630f344f091286a40ae0e9b23` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:47.097334 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1bff9630f344f091286a40ae0e9b23)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:47.099301 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:47.172099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1bff9630f344f091286a40ae0e9b23'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:47.172955 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1bff9630f344f091286a40ae0e9b23'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:47.188304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1bff9630f344f091286a40ae0e9b23'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:47.197308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1bff9630f344f091286a40ae0e9b23'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:47.206667 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1bff9630f344f091286a40ae0e9b23'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:47.212833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1402/2798] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:47 2022] DELETE /identity/v3/users/da1bff9630f344f091286a40ae0e9b23 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:47.217830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d3879-9f2c-4e76-b23e-e33a4d482d9d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:47.218134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d3879-9f2c-4e76-b23e-e33a4d482d9d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:47.218423 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d3879-9f2c-4e76-b23e-e33a4d482d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:47.237953 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-074d3879-9f2c-4e76-b23e-e33a4d482d9d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:47.239210 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-074d3879-9f2c-4e76-b23e-e33a4d482d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:47.315645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1397/2799] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:47.319013 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:47.320518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:47.353597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:47.388930 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9b7e3-a621-4ba1-91aa-0e382d4a52d2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:47.393358 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e279b-6ee6-44de-b6cd-315a342a2d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:47.393852 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e279b-6ee6-44de-b6cd-315a342a2d75 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:47.394141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e279b-6ee6-44de-b6cd-315a342a2d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:47.394419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e279b-6ee6-44de-b6cd-315a342a2d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:47.404461 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e279b-6ee6-44de-b6cd-315a342a2d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:47.406045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e279b-6ee6-44de-b6cd-315a342a2d75 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:47.416804 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1403/2800] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:48.202767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a0eb5-dcfb-4d42-afc7-8c139314d693 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:48.203241 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a0eb5-dcfb-4d42-afc7-8c139314d693 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:48.203390 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a0eb5-dcfb-4d42-afc7-8c139314d693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:48.246002 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:48.258343 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f87a0eb5-dcfb-4d42-afc7-8c139314d693 None None] MFA Rules not processed for user `1578d60cec6f435ab9680b88deff76ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:48.259965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f87a0eb5-dcfb-4d42-afc7-8c139314d693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:48.272656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e279b-6ee6-44de-b6cd-315a342a2d75 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:48.274882 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3f8e279b-6ee6-44de-b6cd-315a342a2d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:48.290901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e279b-6ee6-44de-b6cd-315a342a2d75 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:48.296236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:48.297168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:48.297457 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:48.297739 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] PATH_INFO: `/v3/projects/897eab7ebd3940d483608d9a780a1611` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:48.299045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=897eab7ebd3940d483608d9a780a1611)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:48.301390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:48.366517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897eab7ebd3940d483608d9a780a1611', 'request_id': 'req-5de62f1c-deaa-473c-ad8c-08fdf6793eed'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.376936 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1398/2801] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:48.384006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897eab7ebd3940d483608d9a780a1611', 'request_id': 'req-5de62f1c-deaa-473c-ad8c-08fdf6793eed'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.400283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897eab7ebd3940d483608d9a780a1611', 'request_id': 'req-5de62f1c-deaa-473c-ad8c-08fdf6793eed'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.406178 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a0eb5-dcfb-4d42-afc7-8c139314d693 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:48.408077 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f87a0eb5-dcfb-4d42-afc7-8c139314d693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:48.423720 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] The token cache is being invalidate because project 897eab7ebd3940d483608d9a780a1611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:48.424081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.424703 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.426972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1404/2802] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:48 2022] DELETE /identity/v3/projects/897eab7ebd3940d483608d9a780a1611 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:48.430924 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:48.432398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:48.477677 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a0eb5-dcfb-4d42-afc7-8c139314d693 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:48.528303 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de62f1c-deaa-473c-ad8c-08fdf6793eed admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:48.537362 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:48.538283 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:48.538537 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:48.538880 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] PATH_INFO: `/v3/users/9c0913cf66114b6794e1ebcc8cf42e39` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:48.541937 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c0913cf66114b6794e1ebcc8cf42e39)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:48.544819 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:48.553804 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f2b93-1ffc-407c-9e34-71ba04207037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:48.554460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f2b93-1ffc-407c-9e34-71ba04207037 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:48.554741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f2b93-1ffc-407c-9e34-71ba04207037 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:48.555014 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f2b93-1ffc-407c-9e34-71ba04207037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:48.557569 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-242f2b93-1ffc-407c-9e34-71ba04207037 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:48.631431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c0913cf66114b6794e1ebcc8cf42e39'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.632568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c0913cf66114b6794e1ebcc8cf42e39'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.643131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:48.654958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f2b93-1ffc-407c-9e34-71ba04207037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:48.655193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c0913cf66114b6794e1ebcc8cf42e39'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.657173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f2b93-1ffc-407c-9e34-71ba04207037 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:48.664568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c0913cf66114b6794e1ebcc8cf42e39'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.677538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c0913cf66114b6794e1ebcc8cf42e39'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:48.685619 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1405/2803] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:48 2022] DELETE /identity/v3/users/9c0913cf66114b6794e1ebcc8cf42e39 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:53:48.686767 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1399/2804] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:53:49.023393 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.025005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:49.109138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4239c-6034-41e5-8154-1d6ed29137b5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.114403 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.115335 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.115654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.115936 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] PATH_INFO: `/v3/projects/ba4cbee219a2447cb854c85b3910e8a4` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.117222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba4cbee219a2447cb854c85b3910e8a4)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.119218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.171844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba4cbee219a2447cb854c85b3910e8a4', 'request_id': 'req-520c77ea-248b-4aca-a57f-4a46950d8323'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:49.180654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba4cbee219a2447cb854c85b3910e8a4', 'request_id': 'req-520c77ea-248b-4aca-a57f-4a46950d8323'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:49.186908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba4cbee219a2447cb854c85b3910e8a4', 'request_id': 'req-520c77ea-248b-4aca-a57f-4a46950d8323'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:49.200108 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] The token cache is being invalidate because project ba4cbee219a2447cb854c85b3910e8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:49.200376 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:49.201125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520c77ea-248b-4aca-a57f-4a46950d8323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:49.202733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1406/2805] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:49 2022] DELETE /identity/v3/projects/ba4cbee219a2447cb854c85b3910e8a4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:49.207996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c478a6-2087-4e51-843f-74c812a692b3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.208491 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c478a6-2087-4e51-843f-74c812a692b3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.208656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c478a6-2087-4e51-843f-74c812a692b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.228053 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b2c478a6-2087-4e51-843f-74c812a692b3 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:49.229394 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b2c478a6-2087-4e51-843f-74c812a692b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:49.235741 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:49.292568 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1400/2806] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:49.301568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72aa1c1-4043-4c6c-ab98-b2efa6691623 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.302041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72aa1c1-4043-4c6c-ab98-b2efa6691623 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.302164 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72aa1c1-4043-4c6c-ab98-b2efa6691623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.321041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a72aa1c1-4043-4c6c-ab98-b2efa6691623 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:49.322266 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a72aa1c1-4043-4c6c-ab98-b2efa6691623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:49.330811 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1407/2807] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:49.335217 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c478a6-2087-4e51-843f-74c812a692b3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.345117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c478a6-2087-4e51-843f-74c812a692b3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.350046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822fe3b-1443-4dfc-9d3f-3929dac809c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.350707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822fe3b-1443-4dfc-9d3f-3929dac809c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.350991 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822fe3b-1443-4dfc-9d3f-3929dac809c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.351272 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822fe3b-1443-4dfc-9d3f-3929dac809c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.352062 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822fe3b-1443-4dfc-9d3f-3929dac809c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:49.352385 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822fe3b-1443-4dfc-9d3f-3929dac809c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.353998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822fe3b-1443-4dfc-9d3f-3929dac809c9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.365660 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1401/2808] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:49.370278 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72aa1c1-4043-4c6c-ab98-b2efa6691623 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.379825 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72aa1c1-4043-4c6c-ab98-b2efa6691623 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.385674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ed46b-26b6-46f4-a128-eafc1a605863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.386329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ed46b-26b6-46f4-a128-eafc1a605863 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.386609 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ed46b-26b6-46f4-a128-eafc1a605863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.386878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ed46b-26b6-46f4-a128-eafc1a605863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.387597 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ed46b-26b6-46f4-a128-eafc1a605863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.389511 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ed46b-26b6-46f4-a128-eafc1a605863 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.417612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1408/2809] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:49 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:49.422114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822fe3b-1443-4dfc-9d3f-3929dac809c9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.432306 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822fe3b-1443-4dfc-9d3f-3929dac809c9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.490012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd36316-3c5c-4cf3-9c40-9cbb5f3f2a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.490709 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd36316-3c5c-4cf3-9c40-9cbb5f3f2a6c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.490992 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd36316-3c5c-4cf3-9c40-9cbb5f3f2a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.491271 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd36316-3c5c-4cf3-9c40-9cbb5f3f2a6c admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.492005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd36316-3c5c-4cf3-9c40-9cbb5f3f2a6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.493922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd36316-3c5c-4cf3-9c40-9cbb5f3f2a6c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.517694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1402/2810] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:49 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:49.522419 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ed46b-26b6-46f4-a128-eafc1a605863 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.534134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ed46b-26b6-46f4-a128-eafc1a605863 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.538684 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ca560-8182-4703-8899-97edd5d5436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.539345 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ca560-8182-4703-8899-97edd5d5436d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.540183 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ca560-8182-4703-8899-97edd5d5436d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.540470 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ca560-8182-4703-8899-97edd5d5436d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.541196 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ca560-8182-4703-8899-97edd5d5436d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.542848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ca560-8182-4703-8899-97edd5d5436d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.557724 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1409/2811] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:49.559427 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd36316-3c5c-4cf3-9c40-9cbb5f3f2a6c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.571551 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd36316-3c5c-4cf3-9c40-9cbb5f3f2a6c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.577285 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba3795-1159-4cbb-90a7-6e0385628cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.578068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba3795-1159-4cbb-90a7-6e0385628cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.578373 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba3795-1159-4cbb-90a7-6e0385628cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.578667 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba3795-1159-4cbb-90a7-6e0385628cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.579393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba3795-1159-4cbb-90a7-6e0385628cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.581068 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba3795-1159-4cbb-90a7-6e0385628cc0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.595332 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1403/2812] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:49.596549 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ca560-8182-4703-8899-97edd5d5436d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.607257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ca560-8182-4703-8899-97edd5d5436d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.612178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1df3f-4dfe-4699-97a0-251526917c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.613166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1df3f-4dfe-4699-97a0-251526917c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.613519 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1df3f-4dfe-4699-97a0-251526917c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.613868 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1df3f-4dfe-4699-97a0-251526917c0e admin admin] PATH_INFO: `/v3/projects/e3f43e39e43846fdaaf8f90749c406e7/users/8bc0881029434a94a5745dca20b0fba1/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.627736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1981505-e306-4942-8345-b0ab00bb0478 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.628061 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1981505-e306-4942-8345-b0ab00bb0478 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.628365 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1981505-e306-4942-8345-b0ab00bb0478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.629398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1df3f-4dfe-4699-97a0-251526917c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3f43e39e43846fdaaf8f90749c406e7, user_id=8bc0881029434a94a5745dca20b0fba1, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.632102 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1df3f-4dfe-4699-97a0-251526917c0e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.640762 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:49.647848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1410/2813] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:49 2022] PUT /identity/v3/projects/e3f43e39e43846fdaaf8f90749c406e7/users/8bc0881029434a94a5745dca20b0fba1/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:49.651656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c1981505-e306-4942-8345-b0ab00bb0478 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:49.651656 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c1981505-e306-4942-8345-b0ab00bb0478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:49.658138 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72e7ed-3c03-4417-836c-834c47f4b0f5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.658468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72e7ed-3c03-4417-836c-834c47f4b0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.658734 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72e7ed-3c03-4417-836c-834c47f4b0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.685168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6c72e7ed-3c03-4417-836c-834c47f4b0f5 None None] MFA Rules not processed for user `8bc0881029434a94a5745dca20b0fba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:49.686407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6c72e7ed-3c03-4417-836c-834c47f4b0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:49.710782 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1404/2814] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:53:49.723138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1981505-e306-4942-8345-b0ab00bb0478 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.724751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c1981505-e306-4942-8345-b0ab00bb0478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:49.747489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:49.751248 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1411/2815] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:49.756307 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72e7ed-3c03-4417-836c-834c47f4b0f5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.798117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1981505-e306-4942-8345-b0ab00bb0478 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.803738 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e82f4-03a9-4789-9d6c-f6a37dc89ba1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.804430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e82f4-03a9-4789-9d6c-f6a37dc89ba1 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.804718 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e82f4-03a9-4789-9d6c-f6a37dc89ba1 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.805005 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e82f4-03a9-4789-9d6c-f6a37dc89ba1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.813410 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e82f4-03a9-4789-9d6c-f6a37dc89ba1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.815545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e82f4-03a9-4789-9d6c-f6a37dc89ba1 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.821218 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72e7ed-3c03-4417-836c-834c47f4b0f5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.824044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1405/2816] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:53:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:53:49.826608 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84cc0d-8c15-4ab7-a824-978a9af582a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.827277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84cc0d-8c15-4ab7-a824-978a9af582a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.827533 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84cc0d-8c15-4ab7-a824-978a9af582a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.827854 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84cc0d-8c15-4ab7-a824-978a9af582a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.828420 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84cc0d-8c15-4ab7-a824-978a9af582a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.830431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84cc0d-8c15-4ab7-a824-978a9af582a5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.857957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1412/2817] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:49 2022] POST /identity/v3/projects => generated 401 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:49.862044 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e82f4-03a9-4789-9d6c-f6a37dc89ba1 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.873824 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e82f4-03a9-4789-9d6c-f6a37dc89ba1 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.933309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17090178-a9ab-4437-9cf8-549481e1bb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.933998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17090178-a9ab-4437-9cf8-549481e1bb2a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.934264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17090178-a9ab-4437-9cf8-549481e1bb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.934657 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17090178-a9ab-4437-9cf8-549481e1bb2a admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.935242 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17090178-a9ab-4437-9cf8-549481e1bb2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.937252 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17090178-a9ab-4437-9cf8-549481e1bb2a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.963064 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1406/2818] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:49 2022] POST /identity/v3/users => generated 307 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:49.964111 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84cc0d-8c15-4ab7-a824-978a9af582a5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:49.974817 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84cc0d-8c15-4ab7-a824-978a9af582a5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:49.979575 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7faa90-deaf-458e-b29a-36284295e9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:49.980247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7faa90-deaf-458e-b29a-36284295e9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:49.980535 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7faa90-deaf-458e-b29a-36284295e9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:49.980812 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7faa90-deaf-458e-b29a-36284295e9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:49.981574 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7faa90-deaf-458e-b29a-36284295e9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:49.983189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7faa90-deaf-458e-b29a-36284295e9a6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:49.995136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1413/2819] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:49.999896 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17090178-a9ab-4437-9cf8-549481e1bb2a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.010295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17090178-a9ab-4437-9cf8-549481e1bb2a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.015104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5bb16-3d70-4833-9d08-bdd7ebeeed96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.016136 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5bb16-3d70-4833-9d08-bdd7ebeeed96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.016500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5bb16-3d70-4833-9d08-bdd7ebeeed96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.016855 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5bb16-3d70-4833-9d08-bdd7ebeeed96 admin admin] PATH_INFO: `/v3/projects/7b9fba086479411bb710681f4939c076/users/cae31e01f20c4c58bbb9c9c9df1a9d22/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.041602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5bb16-3d70-4833-9d08-bdd7ebeeed96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9fba086479411bb710681f4939c076, user_id=cae31e01f20c4c58bbb9c9c9df1a9d22, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.044566 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5bb16-3d70-4833-9d08-bdd7ebeeed96 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.064825 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1407/2820] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:49 2022] PUT /identity/v3/projects/7b9fba086479411bb710681f4939c076/users/cae31e01f20c4c58bbb9c9c9df1a9d22/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:50.071755 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7faa90-deaf-458e-b29a-36284295e9a6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.080044 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7faa90-deaf-458e-b29a-36284295e9a6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.089430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19992803-069a-4e32-8f7c-a2589cd11b6e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.089731 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19992803-069a-4e32-8f7c-a2589cd11b6e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.090016 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19992803-069a-4e32-8f7c-a2589cd11b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.110404 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-19992803-069a-4e32-8f7c-a2589cd11b6e None None] MFA Rules not processed for user `1578d60cec6f435ab9680b88deff76ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:50.112482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-19992803-069a-4e32-8f7c-a2589cd11b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:50.156922 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cf41c-cc58-4b2d-bb3f-9c6ef35247af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.157731 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cf41c-cc58-4b2d-bb3f-9c6ef35247af admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.158629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cf41c-cc58-4b2d-bb3f-9c6ef35247af admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.159014 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cf41c-cc58-4b2d-bb3f-9c6ef35247af admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.160155 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cf41c-cc58-4b2d-bb3f-9c6ef35247af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.162517 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cf41c-cc58-4b2d-bb3f-9c6ef35247af admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.165177 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:50.179772 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1414/2821] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:50.179974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1408/2822] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:50.183708 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cf41c-cc58-4b2d-bb3f-9c6ef35247af admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.194590 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cf41c-cc58-4b2d-bb3f-9c6ef35247af admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.199771 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bba50-78be-4e57-b814-7f3163b5648d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.200906 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bba50-78be-4e57-b814-7f3163b5648d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.201264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bba50-78be-4e57-b814-7f3163b5648d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.201629 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bba50-78be-4e57-b814-7f3163b5648d admin admin] PATH_INFO: `/v3/projects/7b9fba086479411bb710681f4939c076/users/cae31e01f20c4c58bbb9c9c9df1a9d22/roles/d8b662bc41c14d36a5b5a94ff86d61ef` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.203988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bba50-78be-4e57-b814-7f3163b5648d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9fba086479411bb710681f4939c076, user_id=cae31e01f20c4c58bbb9c9c9df1a9d22, role_id=d8b662bc41c14d36a5b5a94ff86d61ef)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.206830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bba50-78be-4e57-b814-7f3163b5648d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.218572 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1415/2823] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:50 2022] PUT /identity/v3/projects/7b9fba086479411bb710681f4939c076/users/cae31e01f20c4c58bbb9c9c9df1a9d22/roles/d8b662bc41c14d36a5b5a94ff86d61ef => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:50.229944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cfeac-1f89-46d6-87a9-78daad2af32e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.230481 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cfeac-1f89-46d6-87a9-78daad2af32e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.230594 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cfeac-1f89-46d6-87a9-78daad2af32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.262010 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-509cfeac-1f89-46d6-87a9-78daad2af32e None None] MFA Rules not processed for user `cae31e01f20c4c58bbb9c9c9df1a9d22`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:50.263151 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-509cfeac-1f89-46d6-87a9-78daad2af32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:50.272135 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:50.343768 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1409/2824] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:50.356273 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bba50-78be-4e57-b814-7f3163b5648d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.358782 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4c5bba50-78be-4e57-b814-7f3163b5648d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:50.409685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7408c-b0d6-48fe-9d9c-c8d85e8ada43 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.410631 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7408c-b0d6-48fe-9d9c-c8d85e8ada43 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.410986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7408c-b0d6-48fe-9d9c-c8d85e8ada43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.426503 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bba50-78be-4e57-b814-7f3163b5648d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.429944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-90d7408c-b0d6-48fe-9d9c-c8d85e8ada43 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:50.430875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913f176-a6db-4357-8c82-1e95b1cdbe48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.431393 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-90d7408c-b0d6-48fe-9d9c-c8d85e8ada43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:50.431591 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913f176-a6db-4357-8c82-1e95b1cdbe48 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.431840 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913f176-a6db-4357-8c82-1e95b1cdbe48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.432115 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913f176-a6db-4357-8c82-1e95b1cdbe48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.443616 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913f176-a6db-4357-8c82-1e95b1cdbe48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.445079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913f176-a6db-4357-8c82-1e95b1cdbe48 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.467712 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1416/2825] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:50 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:50.500640 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1410/2826] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 13 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:50.514645 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913f176-a6db-4357-8c82-1e95b1cdbe48 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.543384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7408c-b0d6-48fe-9d9c-c8d85e8ada43 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.554259 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7408c-b0d6-48fe-9d9c-c8d85e8ada43 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.558009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286deb1e-9b63-451d-bb1a-ad55f4581bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.558664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286deb1e-9b63-451d-bb1a-ad55f4581bee service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.558949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286deb1e-9b63-451d-bb1a-ad55f4581bee service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.559248 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286deb1e-9b63-451d-bb1a-ad55f4581bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.590453 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913f176-a6db-4357-8c82-1e95b1cdbe48 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.595814 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2167a7-c9ad-48e5-93fa-d6466dba66fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.596502 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2167a7-c9ad-48e5-93fa-d6466dba66fa service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.596833 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2167a7-c9ad-48e5-93fa-d6466dba66fa service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.597109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2167a7-c9ad-48e5-93fa-d6466dba66fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.605835 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2167a7-c9ad-48e5-93fa-d6466dba66fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.607442 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2167a7-c9ad-48e5-93fa-d6466dba66fa service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.616942 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1417/2827] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:53:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:53:50.629177 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286deb1e-9b63-451d-bb1a-ad55f4581bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.631031 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286deb1e-9b63-451d-bb1a-ad55f4581bee service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.640746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1411/2828] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:50.864948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b6f82-2f26-4234-8103-94d9ee4ab747 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.864948 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b6f82-2f26-4234-8103-94d9ee4ab747 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.865414 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b6f82-2f26-4234-8103-94d9ee4ab747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.886198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cc8b6f82-2f26-4234-8103-94d9ee4ab747 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:50.887513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cc8b6f82-2f26-4234-8103-94d9ee4ab747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:50.895041 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1418/2829] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 13 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:50.910575 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286deb1e-9b63-451d-bb1a-ad55f4581bee service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.924937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286deb1e-9b63-451d-bb1a-ad55f4581bee service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.931338 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a96c9c-50dd-4d3b-9133-e01c319c409c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.932264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a96c9c-50dd-4d3b-9133-e01c319c409c service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.932636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a96c9c-50dd-4d3b-9133-e01c319c409c service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.933350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a96c9c-50dd-4d3b-9133-e01c319c409c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.949515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a96c9c-50dd-4d3b-9133-e01c319c409c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.952243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a96c9c-50dd-4d3b-9133-e01c319c409c service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.964655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1412/2830] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:53:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:53:51.330784 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77fe4d-15e0-4c99-8bb4-68a7250999d2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:51.331073 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77fe4d-15e0-4c99-8bb4-68a7250999d2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:51.331348 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77fe4d-15e0-4c99-8bb4-68a7250999d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:51.349490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8e77fe4d-15e0-4c99-8bb4-68a7250999d2 None None] MFA Rules not processed for user `95dcfd3d412a47298d7d5cf7b3b56f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:51.350701 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8e77fe4d-15e0-4c99-8bb4-68a7250999d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:51.358395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1419/2831] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 13 20:53:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:51.370784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a96c9c-50dd-4d3b-9133-e01c319c409c service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:51.382039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a96c9c-50dd-4d3b-9133-e01c319c409c service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:51.387540 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb4d39-723a-4e98-a740-355c6b971fbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f27793f58c74ffb9a7a8e4cdddb8169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:51.388195 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb4d39-723a-4e98-a740-355c6b971fbf service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:51.388482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb4d39-723a-4e98-a740-355c6b971fbf service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:51.388764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb4d39-723a-4e98-a740-355c6b971fbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:51.397164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb4d39-723a-4e98-a740-355c6b971fbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:51.398760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb4d39-723a-4e98-a740-355c6b971fbf service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:51.407282 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1413/2832] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 13 20:53:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:53:53.579327 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77fe4d-15e0-4c99-8bb4-68a7250999d2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:53.581883 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8e77fe4d-15e0-4c99-8bb4-68a7250999d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:53.658984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77fe4d-15e0-4c99-8bb4-68a7250999d2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:53.664745 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:53.665665 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:53.666151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:53.666434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] PATH_INFO: `/v3/users/79db66d78174491cbc08384153711601` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:53.667690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79db66d78174491cbc08384153711601)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:53.669844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:53.675732 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:53.728826 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79db66d78174491cbc08384153711601'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:53.729497 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79db66d78174491cbc08384153711601'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:53.742386 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79db66d78174491cbc08384153711601'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:53.749368 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79db66d78174491cbc08384153711601'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:53.757219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79db66d78174491cbc08384153711601'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:53.763132 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1420/2833] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:53 2022] DELETE /identity/v3/users/79db66d78174491cbc08384153711601 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:53.767736 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b257fc-82c5-459b-a135-9d31132a690f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:53.768212 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b257fc-82c5-459b-a135-9d31132a690f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:53.768374 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b257fc-82c5-459b-a135-9d31132a690f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:53.786455 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-16b257fc-82c5-459b-a135-9d31132a690f None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:53.787970 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-16b257fc-82c5-459b-a135-9d31132a690f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:53.804441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:53.847173 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1414/2834] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:53.862965 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:53.864541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:53.928007 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64ec9d-1baf-4814-8592-ef6811b2e170 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:53.931839 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce68750-9d5f-4888-98f0-bfd0368bd3c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:53.932538 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce68750-9d5f-4888-98f0-bfd0368bd3c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:53.932829 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce68750-9d5f-4888-98f0-bfd0368bd3c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:53.933532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce68750-9d5f-4888-98f0-bfd0368bd3c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:53.943411 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce68750-9d5f-4888-98f0-bfd0368bd3c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:53.945055 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce68750-9d5f-4888-98f0-bfd0368bd3c3 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:53.955262 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1421/2835] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:54.141141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b257fc-82c5-459b-a135-9d31132a690f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:54.144081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-16b257fc-82c5-459b-a135-9d31132a690f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:54.163085 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b257fc-82c5-459b-a135-9d31132a690f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:54.170445 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:54.171309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:54.171712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:54.171891 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] PATH_INFO: `/v3/projects/6a690889825843b2b7db3f87806d9cfd` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:54.173278 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a690889825843b2b7db3f87806d9cfd)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:54.175960 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:54.234897 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a690889825843b2b7db3f87806d9cfd', 'request_id': 'req-dd70198c-9d92-4557-8cd8-495fa025877c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:54.241545 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a690889825843b2b7db3f87806d9cfd', 'request_id': 'req-dd70198c-9d92-4557-8cd8-495fa025877c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:54.247773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a690889825843b2b7db3f87806d9cfd', 'request_id': 'req-dd70198c-9d92-4557-8cd8-495fa025877c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:54.266723 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] The token cache is being invalidate because project 6a690889825843b2b7db3f87806d9cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:54.267091 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:54.268092 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-dd70198c-9d92-4557-8cd8-495fa025877c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:54.270494 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1415/2836] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:54 2022] DELETE /identity/v3/projects/6a690889825843b2b7db3f87806d9cfd => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:54.880892 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce68750-9d5f-4888-98f0-bfd0368bd3c3 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:54.882496 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9ce68750-9d5f-4888-98f0-bfd0368bd3c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:54.895802 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce68750-9d5f-4888-98f0-bfd0368bd3c3 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:54.958425 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:54.959674 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:54.959881 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:54.960172 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] PATH_INFO: `/v3/users/8bc0881029434a94a5745dca20b0fba1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:54.961553 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc0881029434a94a5745dca20b0fba1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:54.964628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:54.998157 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:55.019877 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc0881029434a94a5745dca20b0fba1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.020612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc0881029434a94a5745dca20b0fba1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.034168 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc0881029434a94a5745dca20b0fba1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.041677 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc0881029434a94a5745dca20b0fba1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.051069 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc0881029434a94a5745dca20b0fba1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.058271 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1422/2837] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:54 2022] DELETE /identity/v3/users/8bc0881029434a94a5745dca20b0fba1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:55.064916 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7e36c-44d2-4c9f-99f0-eb2c374a7d39 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:55.065693 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7e36c-44d2-4c9f-99f0-eb2c374a7d39 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:55.065814 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7e36c-44d2-4c9f-99f0-eb2c374a7d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:55.083439 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b3e7e36c-44d2-4c9f-99f0-eb2c374a7d39 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:55.084706 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b3e7e36c-44d2-4c9f-99f0-eb2c374a7d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:55.152101 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1416/2838] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:55.162247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:55.163956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:55.224233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5c215-a495-4e63-92d2-7ed9ddd2de73 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:55.228380 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de6529-5c65-4d23-8b8f-729a9f837618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:55.229046 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de6529-5c65-4d23-8b8f-729a9f837618 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:55.229333 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de6529-5c65-4d23-8b8f-729a9f837618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:55.229612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de6529-5c65-4d23-8b8f-729a9f837618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:55.239325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de6529-5c65-4d23-8b8f-729a9f837618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:55.240828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de6529-5c65-4d23-8b8f-729a9f837618 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:55.250485 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1423/2839] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:55.585193 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7e36c-44d2-4c9f-99f0-eb2c374a7d39 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:55.586918 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b3e7e36c-44d2-4c9f-99f0-eb2c374a7d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:55.600949 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7e36c-44d2-4c9f-99f0-eb2c374a7d39 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:55.613482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:55.614505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:55.614860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:55.615208 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] PATH_INFO: `/v3/projects/e3f43e39e43846fdaaf8f90749c406e7` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:55.616648 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3f43e39e43846fdaaf8f90749c406e7)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:55.620116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:55.676847 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:55.684088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3f43e39e43846fdaaf8f90749c406e7', 'request_id': 'req-56a02c4c-980b-46fa-a8dc-6249ac57da20'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.692948 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3f43e39e43846fdaaf8f90749c406e7', 'request_id': 'req-56a02c4c-980b-46fa-a8dc-6249ac57da20'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.699243 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3f43e39e43846fdaaf8f90749c406e7', 'request_id': 'req-56a02c4c-980b-46fa-a8dc-6249ac57da20'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.712166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] The token cache is being invalidate because project e3f43e39e43846fdaaf8f90749c406e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:55.712460 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.713088 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.714652 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1417/2840] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:55 2022] DELETE /identity/v3/projects/e3f43e39e43846fdaaf8f90749c406e7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:55.718790 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de6529-5c65-4d23-8b8f-729a9f837618 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:55.729636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de6529-5c65-4d23-8b8f-729a9f837618 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:55.788917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:55.789808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:55.790090 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:55.790391 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] PATH_INFO: `/v3/users/cae31e01f20c4c58bbb9c9c9df1a9d22` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:55.791648 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae31e01f20c4c58bbb9c9c9df1a9d22)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:55.793612 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:55.842221 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae31e01f20c4c58bbb9c9c9df1a9d22'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.842799 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae31e01f20c4c58bbb9c9c9df1a9d22'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.844725 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:55.857005 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae31e01f20c4c58bbb9c9c9df1a9d22'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.864159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae31e01f20c4c58bbb9c9c9df1a9d22'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.873481 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f4366095-a7bd-43e0-ba9d-ffefa6e06646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae31e01f20c4c58bbb9c9c9df1a9d22'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:55.880861 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1424/2841] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:55 2022] DELETE /identity/v3/users/cae31e01f20c4c58bbb9c9c9df1a9d22 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:56.326228 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:56.327900 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:56.350155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02c4c-980b-46fa-a8dc-6249ac57da20 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:56.415645 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:56.416661 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.416980 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.417328 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] PATH_INFO: `/v3/projects/7b9fba086479411bb710681f4939c076` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:56.418641 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b9fba086479411bb710681f4939c076)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:56.420746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:56.471276 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b9fba086479411bb710681f4939c076', 'request_id': 'req-e41bec38-a568-46a2-be51-70376fc8ceec'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:56.478434 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b9fba086479411bb710681f4939c076', 'request_id': 'req-e41bec38-a568-46a2-be51-70376fc8ceec'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:56.484721 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b9fba086479411bb710681f4939c076', 'request_id': 'req-e41bec38-a568-46a2-be51-70376fc8ceec'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:56.498341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] The token cache is being invalidate because project 7b9fba086479411bb710681f4939c076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:56.498636 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:56.499308 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e41bec38-a568-46a2-be51-70376fc8ceec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:56.500899 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1418/2842] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:56 2022] DELETE /identity/v3/projects/7b9fba086479411bb710681f4939c076 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:56.506342 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f30fc-e29a-4cdd-baf4-9aed6afc9dc7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.506805 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f30fc-e29a-4cdd-baf4-9aed6afc9dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.506923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f30fc-e29a-4cdd-baf4-9aed6afc9dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:56.525668 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-923f30fc-e29a-4cdd-baf4-9aed6afc9dc7 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:56.526891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-923f30fc-e29a-4cdd-baf4-9aed6afc9dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:56.586500 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1425/2843] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:56.594539 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7363126-81bb-454f-8b1a-cfd476fb42a4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.594830 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7363126-81bb-454f-8b1a-cfd476fb42a4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.595115 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7363126-81bb-454f-8b1a-cfd476fb42a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:56.617793 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a7363126-81bb-454f-8b1a-cfd476fb42a4 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:56.619001 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a7363126-81bb-454f-8b1a-cfd476fb42a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:56.631939 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1419/2844] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:56.636964 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f30fc-e29a-4cdd-baf4-9aed6afc9dc7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:56.651939 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f30fc-e29a-4cdd-baf4-9aed6afc9dc7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:56.659970 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7a3da-5b99-4545-be30-7d1ca2442bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:56.660637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7a3da-5b99-4545-be30-7d1ca2442bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.660894 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7a3da-5b99-4545-be30-7d1ca2442bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.661181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7a3da-5b99-4545-be30-7d1ca2442bc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:56.661928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7a3da-5b99-4545-be30-7d1ca2442bc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:56.662281 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7a3da-5b99-4545-be30-7d1ca2442bc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:56.663963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7a3da-5b99-4545-be30-7d1ca2442bc1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:56.675957 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1426/2845] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:56.680376 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7363126-81bb-454f-8b1a-cfd476fb42a4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:56.690166 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7363126-81bb-454f-8b1a-cfd476fb42a4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:56.696117 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b121da-7374-48fe-a015-87b0b0e9d270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:56.696785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b121da-7374-48fe-a015-87b0b0e9d270 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.697071 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b121da-7374-48fe-a015-87b0b0e9d270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.697350 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b121da-7374-48fe-a015-87b0b0e9d270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:56.698052 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b121da-7374-48fe-a015-87b0b0e9d270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:56.699967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b121da-7374-48fe-a015-87b0b0e9d270 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:56.729309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1420/2846] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:56 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:56.734032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7a3da-5b99-4545-be30-7d1ca2442bc1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:56.744436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7a3da-5b99-4545-be30-7d1ca2442bc1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:56.802030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55b312-540b-4935-9e7d-7d714ebe8813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:56.802717 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55b312-540b-4935-9e7d-7d714ebe8813 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.803002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55b312-540b-4935-9e7d-7d714ebe8813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.803280 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55b312-540b-4935-9e7d-7d714ebe8813 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:56.804008 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55b312-540b-4935-9e7d-7d714ebe8813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:56.805944 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55b312-540b-4935-9e7d-7d714ebe8813 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:56.833862 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1427/2847] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:56 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:56.837888 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b121da-7374-48fe-a015-87b0b0e9d270 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:56.848362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b121da-7374-48fe-a015-87b0b0e9d270 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:56.852984 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304116b2-a84e-4e2a-8e06-5777a6e92adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:56.853665 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304116b2-a84e-4e2a-8e06-5777a6e92adb admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.853988 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304116b2-a84e-4e2a-8e06-5777a6e92adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.854250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304116b2-a84e-4e2a-8e06-5777a6e92adb admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:56.855086 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304116b2-a84e-4e2a-8e06-5777a6e92adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:56.856996 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304116b2-a84e-4e2a-8e06-5777a6e92adb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:56.868025 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1421/2848] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:56.871890 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55b312-540b-4935-9e7d-7d714ebe8813 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:56.881995 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55b312-540b-4935-9e7d-7d714ebe8813 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:56.886978 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ccb55-2eeb-4ead-97c8-c4d51abb9e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:56.887670 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ccb55-2eeb-4ead-97c8-c4d51abb9e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.887952 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ccb55-2eeb-4ead-97c8-c4d51abb9e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.888228 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ccb55-2eeb-4ead-97c8-c4d51abb9e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:56.888950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ccb55-2eeb-4ead-97c8-c4d51abb9e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:56.890634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ccb55-2eeb-4ead-97c8-c4d51abb9e1c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:56.901252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1428/2849] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:53:56.905478 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304116b2-a84e-4e2a-8e06-5777a6e92adb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:56.916289 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304116b2-a84e-4e2a-8e06-5777a6e92adb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:56.922000 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be016d0c-08bc-44b1-8bce-cd1509cbb278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:56.922924 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be016d0c-08bc-44b1-8bce-cd1509cbb278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.923211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be016d0c-08bc-44b1-8bce-cd1509cbb278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.923485 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be016d0c-08bc-44b1-8bce-cd1509cbb278 admin admin] PATH_INFO: `/v3/projects/036e910d5ea64cbe953e739319db7eb2/users/d73dea133cbd48c3856d1bc198e0d633/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:56.939106 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be016d0c-08bc-44b1-8bce-cd1509cbb278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036e910d5ea64cbe953e739319db7eb2, user_id=d73dea133cbd48c3856d1bc198e0d633, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:56.942164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be016d0c-08bc-44b1-8bce-cd1509cbb278 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:56.945258 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:56.961694 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1422/2850] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:56 2022] PUT /identity/v3/projects/036e910d5ea64cbe953e739319db7eb2/users/d73dea133cbd48c3856d1bc198e0d633/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:56.972330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6416ce-0ed7-4fee-9437-d312a5ee73f8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:56.972642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6416ce-0ed7-4fee-9437-d312a5ee73f8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:56.972908 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6416ce-0ed7-4fee-9437-d312a5ee73f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:57.001632 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-df6416ce-0ed7-4fee-9437-d312a5ee73f8 None None] MFA Rules not processed for user `d73dea133cbd48c3856d1bc198e0d633`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:57.002866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-df6416ce-0ed7-4fee-9437-d312a5ee73f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:57.024145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:57.074120 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1429/2851] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:57.085990 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be016d0c-08bc-44b1-8bce-cd1509cbb278 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:57.088121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-be016d0c-08bc-44b1-8bce-cd1509cbb278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:57.148213 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be016d0c-08bc-44b1-8bce-cd1509cbb278 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:57.152675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d0d96-d16b-4fdf-ac6c-ac9313b6618f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:57.153362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d0d96-d16b-4fdf-ac6c-ac9313b6618f service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:57.153644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d0d96-d16b-4fdf-ac6c-ac9313b6618f service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:57.153922 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d0d96-d16b-4fdf-ac6c-ac9313b6618f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:57.163267 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d0d96-d16b-4fdf-ac6c-ac9313b6618f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:57.164757 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d0d96-d16b-4fdf-ac6c-ac9313b6618f service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:57.173734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1423/2852] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:57.195279 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6416ce-0ed7-4fee-9437-d312a5ee73f8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:57.272045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6416ce-0ed7-4fee-9437-d312a5ee73f8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:57.277212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:57.278159 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:57.278436 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:57.278715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] PATH_INFO: `/v3/users/d73dea133cbd48c3856d1bc198e0d633` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:57.280070 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73dea133cbd48c3856d1bc198e0d633)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:57.282131 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:57.336130 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73dea133cbd48c3856d1bc198e0d633'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.336824 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73dea133cbd48c3856d1bc198e0d633'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.348515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73dea133cbd48c3856d1bc198e0d633'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.355600 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73dea133cbd48c3856d1bc198e0d633'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.364859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73dea133cbd48c3856d1bc198e0d633'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.371227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1430/2853] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:57 2022] DELETE /identity/v3/users/d73dea133cbd48c3856d1bc198e0d633 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:57.376137 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ef02-151f-44de-85b9-5bc74ec5fb6b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:57.376644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ef02-151f-44de-85b9-5bc74ec5fb6b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:57.376746 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ef02-151f-44de-85b9-5bc74ec5fb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:57.395194 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c259ef02-151f-44de-85b9-5bc74ec5fb6b None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:57.396482 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c259ef02-151f-44de-85b9-5bc74ec5fb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:57.459134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1424/2854] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:57.469006 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:57.470475 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:57.517827 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:57.530950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b78480-a104-4e4e-996c-32606edcec26 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:57.535030 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63eebf4-93c6-419d-a199-33ff14b81536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:57.535726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63eebf4-93c6-419d-a199-33ff14b81536 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:57.536019 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63eebf4-93c6-419d-a199-33ff14b81536 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:57.536304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63eebf4-93c6-419d-a199-33ff14b81536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:57.547344 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63eebf4-93c6-419d-a199-33ff14b81536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:57.548831 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63eebf4-93c6-419d-a199-33ff14b81536 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:57.558595 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1431/2855] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:57.847851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ef02-151f-44de-85b9-5bc74ec5fb6b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:57.849453 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c259ef02-151f-44de-85b9-5bc74ec5fb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:57.873330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ef02-151f-44de-85b9-5bc74ec5fb6b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:57.880143 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:57.888141 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:57.888409 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:57.888687 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] PATH_INFO: `/v3/projects/036e910d5ea64cbe953e739319db7eb2` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:57.890813 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] RBAC: Authorizing `identity:delete_project(project_id=036e910d5ea64cbe953e739319db7eb2)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:57.892805 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:57.943957 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036e910d5ea64cbe953e739319db7eb2', 'request_id': 'req-4db1066c-5fc2-4564-99c8-af7efa44d39a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.951795 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036e910d5ea64cbe953e739319db7eb2', 'request_id': 'req-4db1066c-5fc2-4564-99c8-af7efa44d39a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.958037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036e910d5ea64cbe953e739319db7eb2', 'request_id': 'req-4db1066c-5fc2-4564-99c8-af7efa44d39a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.971875 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] The token cache is being invalidate because project 036e910d5ea64cbe953e739319db7eb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:57.972408 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.973039 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4db1066c-5fc2-4564-99c8-af7efa44d39a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:57.974753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1425/2856] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:57 2022] DELETE /identity/v3/projects/036e910d5ea64cbe953e739319db7eb2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:57.979912 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90994361-07a3-4d34-9277-0c60c2f4655c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:57.980208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90994361-07a3-4d34-9277-0c60c2f4655c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:57.980650 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90994361-07a3-4d34-9277-0c60c2f4655c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:57.999045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-90994361-07a3-4d34-9277-0c60c2f4655c None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:58.000304 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-90994361-07a3-4d34-9277-0c60c2f4655c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:58.071094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1432/2857] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:58.080040 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff635b1e-1e26-4803-9da8-ca3c355568f3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.080428 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff635b1e-1e26-4803-9da8-ca3c355568f3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.080772 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff635b1e-1e26-4803-9da8-ca3c355568f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.100004 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ff635b1e-1e26-4803-9da8-ca3c355568f3 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:58.101204 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ff635b1e-1e26-4803-9da8-ca3c355568f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:58.109079 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1426/2858] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:58.113353 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90994361-07a3-4d34-9277-0c60c2f4655c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:58.123001 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90994361-07a3-4d34-9277-0c60c2f4655c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:58.127555 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c63b16-60ed-41fe-9246-7c7b46586f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:58.128277 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c63b16-60ed-41fe-9246-7c7b46586f69 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.128567 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c63b16-60ed-41fe-9246-7c7b46586f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.128844 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c63b16-60ed-41fe-9246-7c7b46586f69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.129566 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c63b16-60ed-41fe-9246-7c7b46586f69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:58.129875 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c63b16-60ed-41fe-9246-7c7b46586f69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:58.131484 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c63b16-60ed-41fe-9246-7c7b46586f69 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:58.142741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1433/2859] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:58.146998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff635b1e-1e26-4803-9da8-ca3c355568f3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:58.156403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff635b1e-1e26-4803-9da8-ca3c355568f3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:58.161666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e897b8-548b-47cc-9d1e-81e74bc44da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:58.162312 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e897b8-548b-47cc-9d1e-81e74bc44da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.162595 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e897b8-548b-47cc-9d1e-81e74bc44da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.162909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e897b8-548b-47cc-9d1e-81e74bc44da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.163653 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e897b8-548b-47cc-9d1e-81e74bc44da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:58.165561 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e897b8-548b-47cc-9d1e-81e74bc44da4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:58.178858 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:58.193535 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1427/2860] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:58 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:58.197986 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c63b16-60ed-41fe-9246-7c7b46586f69 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:58.207928 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c63b16-60ed-41fe-9246-7c7b46586f69 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:58.273628 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3869178-6266-4f2b-95a5-30acffa5e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:58.274308 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3869178-6266-4f2b-95a5-30acffa5e28c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.274594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3869178-6266-4f2b-95a5-30acffa5e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.274871 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3869178-6266-4f2b-95a5-30acffa5e28c admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.275654 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3869178-6266-4f2b-95a5-30acffa5e28c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:58.277606 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3869178-6266-4f2b-95a5-30acffa5e28c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:58.299208 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1434/2861] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:58 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:58.303985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e897b8-548b-47cc-9d1e-81e74bc44da4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:58.319500 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e897b8-548b-47cc-9d1e-81e74bc44da4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:58.328262 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc021e8-b767-4113-b89a-07b58dd45f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:58.329189 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc021e8-b767-4113-b89a-07b58dd45f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.329458 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc021e8-b767-4113-b89a-07b58dd45f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.329857 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc021e8-b767-4113-b89a-07b58dd45f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.330864 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc021e8-b767-4113-b89a-07b58dd45f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:58.332612 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc021e8-b767-4113-b89a-07b58dd45f04 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:58.343704 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1428/2862] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:58.347828 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3869178-6266-4f2b-95a5-30acffa5e28c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:58.358370 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3869178-6266-4f2b-95a5-30acffa5e28c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:58.363514 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33381c0-a7c2-48c7-a88e-28981521c828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:58.364431 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33381c0-a7c2-48c7-a88e-28981521c828 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.364917 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33381c0-a7c2-48c7-a88e-28981521c828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.366390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33381c0-a7c2-48c7-a88e-28981521c828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.366390 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33381c0-a7c2-48c7-a88e-28981521c828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:58.367744 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33381c0-a7c2-48c7-a88e-28981521c828 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:58.377991 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1435/2863] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:58.382149 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc021e8-b767-4113-b89a-07b58dd45f04 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:58.393295 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc021e8-b767-4113-b89a-07b58dd45f04 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:58.399051 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603f478-ab5d-4e3a-8400-93b83b59d009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:58.400023 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f478-ab5d-4e3a-8400-93b83b59d009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.400302 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f478-ab5d-4e3a-8400-93b83b59d009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.400573 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f478-ab5d-4e3a-8400-93b83b59d009 admin admin] PATH_INFO: `/v3/projects/870f1c62562a4f1facc2a74169786d1b/users/5e1be23962e54f45b858027b2d3b5fcd/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.417206 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603f478-ab5d-4e3a-8400-93b83b59d009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=870f1c62562a4f1facc2a74169786d1b, user_id=5e1be23962e54f45b858027b2d3b5fcd, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:58.419986 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603f478-ab5d-4e3a-8400-93b83b59d009 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:58.434938 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1429/2864] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:58 2022] PUT /identity/v3/projects/870f1c62562a4f1facc2a74169786d1b/users/5e1be23962e54f45b858027b2d3b5fcd/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:53:58.445736 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d01a1-9f60-45c0-985f-74f0ffbf448d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.446032 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d01a1-9f60-45c0-985f-74f0ffbf448d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.446312 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d01a1-9f60-45c0-985f-74f0ffbf448d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.472958 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-586d01a1-9f60-45c0-985f-74f0ffbf448d None None] MFA Rules not processed for user `5e1be23962e54f45b858027b2d3b5fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:58.474163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-586d01a1-9f60-45c0-985f-74f0ffbf448d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:58.513539 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:58.540463 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1436/2865] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:58.551839 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603f478-ab5d-4e3a-8400-93b83b59d009 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:58.553658 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1603f478-ab5d-4e3a-8400-93b83b59d009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:58.620818 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603f478-ab5d-4e3a-8400-93b83b59d009 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:58.625696 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e0050-2295-49eb-b761-f027c956e0e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:58.626712 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e0050-2295-49eb-b761-f027c956e0e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.627361 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e0050-2295-49eb-b761-f027c956e0e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.627909 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e0050-2295-49eb-b761-f027c956e0e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.637563 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e0050-2295-49eb-b761-f027c956e0e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:58.639017 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e0050-2295-49eb-b761-f027c956e0e9 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:58.648508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1430/2866] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:58.668999 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d01a1-9f60-45c0-985f-74f0ffbf448d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:58.740770 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d01a1-9f60-45c0-985f-74f0ffbf448d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:58.745860 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:58.746748 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.747036 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.747314 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] PATH_INFO: `/v3/users/5e1be23962e54f45b858027b2d3b5fcd` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.748598 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e1be23962e54f45b858027b2d3b5fcd)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:58.750550 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:58.798152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e1be23962e54f45b858027b2d3b5fcd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:58.798796 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e1be23962e54f45b858027b2d3b5fcd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:58.810398 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e1be23962e54f45b858027b2d3b5fcd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:58.817848 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e1be23962e54f45b858027b2d3b5fcd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:58.825571 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e1be23962e54f45b858027b2d3b5fcd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:58.831559 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1437/2867] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:53:58 2022] DELETE /identity/v3/users/5e1be23962e54f45b858027b2d3b5fcd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:58.836277 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925a54f-569e-48ac-a041-9030ddd3d1c5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:58.836586 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925a54f-569e-48ac-a041-9030ddd3d1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:58.836863 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925a54f-569e-48ac-a041-9030ddd3d1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:58.854870 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f925a54f-569e-48ac-a041-9030ddd3d1c5 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:58.856099 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f925a54f-569e-48ac-a041-9030ddd3d1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:58.917348 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1431/2868] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:58.927318 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:58.928961 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:58.997299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696aea9a-5b35-4913-84c5-86e750c72526 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:59.002088 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eda7f-a036-495e-8e1c-f2f2a2139935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:59.002793 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eda7f-a036-495e-8e1c-f2f2a2139935 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.003129 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eda7f-a036-495e-8e1c-f2f2a2139935 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.003417 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eda7f-a036-495e-8e1c-f2f2a2139935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.014842 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eda7f-a036-495e-8e1c-f2f2a2139935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:59.016330 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eda7f-a036-495e-8e1c-f2f2a2139935 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:59.026165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1438/2869] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:53:59.309104 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925a54f-569e-48ac-a041-9030ddd3d1c5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:59.310589 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f925a54f-569e-48ac-a041-9030ddd3d1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:59.324993 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925a54f-569e-48ac-a041-9030ddd3d1c5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:59.330264 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:59.331642 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.331908 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.332223 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] PATH_INFO: `/v3/projects/870f1c62562a4f1facc2a74169786d1b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.333430 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] RBAC: Authorizing `identity:delete_project(project_id=870f1c62562a4f1facc2a74169786d1b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:59.335407 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:59.385330 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '870f1c62562a4f1facc2a74169786d1b', 'request_id': 'req-1b63f84e-5404-4ee2-89ce-3ae123df754f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:59.391655 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '870f1c62562a4f1facc2a74169786d1b', 'request_id': 'req-1b63f84e-5404-4ee2-89ce-3ae123df754f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:59.393791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:59.398931 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '870f1c62562a4f1facc2a74169786d1b', 'request_id': 'req-1b63f84e-5404-4ee2-89ce-3ae123df754f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:59.412114 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] The token cache is being invalidate because project 870f1c62562a4f1facc2a74169786d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:53:59.412399 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:59.412998 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1b63f84e-5404-4ee2-89ce-3ae123df754f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:53:59.414549 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1432/2870] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:53:59 2022] DELETE /identity/v3/projects/870f1c62562a4f1facc2a74169786d1b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:59.419611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58879015-b1e1-467b-9a70-92131d73a466 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.419907 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58879015-b1e1-467b-9a70-92131d73a466 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.420193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58879015-b1e1-467b-9a70-92131d73a466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.426568 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:59.441378 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-58879015-b1e1-467b-9a70-92131d73a466 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:59.442637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-58879015-b1e1-467b-9a70-92131d73a466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:59.501375 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1439/2871] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:59.509548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f652a1b-9cda-4feb-aa32-1b34fcf6124d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.509791 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f652a1b-9cda-4feb-aa32-1b34fcf6124d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.510069 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f652a1b-9cda-4feb-aa32-1b34fcf6124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.529604 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1f652a1b-9cda-4feb-aa32-1b34fcf6124d None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:59.530860 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1f652a1b-9cda-4feb-aa32-1b34fcf6124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:59.540205 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1433/2872] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:53:59.544743 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58879015-b1e1-467b-9a70-92131d73a466 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:59.554702 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58879015-b1e1-467b-9a70-92131d73a466 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:59.560569 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442e8dc-8047-40bf-a692-87e39ed3a6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:59.561233 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442e8dc-8047-40bf-a692-87e39ed3a6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.561513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442e8dc-8047-40bf-a692-87e39ed3a6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.561790 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442e8dc-8047-40bf-a692-87e39ed3a6b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.562518 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442e8dc-8047-40bf-a692-87e39ed3a6b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:59.562809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442e8dc-8047-40bf-a692-87e39ed3a6b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:59.564491 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442e8dc-8047-40bf-a692-87e39ed3a6b3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:59.576715 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1440/2873] 10.222.0.47 () {58 vars in 1152 bytes} [Wed Jul 13 20:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:59.581134 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f652a1b-9cda-4feb-aa32-1b34fcf6124d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:59.590646 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f652a1b-9cda-4feb-aa32-1b34fcf6124d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:59.595823 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65d56-5a9c-4105-950f-9e781a719d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:59.596495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65d56-5a9c-4105-950f-9e781a719d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.596835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65d56-5a9c-4105-950f-9e781a719d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.597113 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65d56-5a9c-4105-950f-9e781a719d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.597815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c65d56-5a9c-4105-950f-9e781a719d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:59.599764 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c65d56-5a9c-4105-950f-9e781a719d36 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:59.626982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1434/2874] 10.222.0.47 () {60 vars in 1152 bytes} [Wed Jul 13 20:53:59 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:59.631360 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442e8dc-8047-40bf-a692-87e39ed3a6b3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:59.642920 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442e8dc-8047-40bf-a692-87e39ed3a6b3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:59.702678 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447150ca-673b-43d3-a879-dc16945210a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:59.703351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447150ca-673b-43d3-a879-dc16945210a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.703637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447150ca-673b-43d3-a879-dc16945210a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.703945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447150ca-673b-43d3-a879-dc16945210a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.704627 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447150ca-673b-43d3-a879-dc16945210a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:59.706605 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447150ca-673b-43d3-a879-dc16945210a3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:59.728486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1441/2875] 10.222.0.47 () {60 vars in 1143 bytes} [Wed Jul 13 20:53:59 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:59.732985 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65d56-5a9c-4105-950f-9e781a719d36 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:59.744303 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65d56-5a9c-4105-950f-9e781a719d36 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:59.749666 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492651b-e8f6-430a-936f-3fa6cc4ff064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:59.750341 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492651b-e8f6-430a-936f-3fa6cc4ff064 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.750632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492651b-e8f6-430a-936f-3fa6cc4ff064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.750913 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492651b-e8f6-430a-936f-3fa6cc4ff064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.751680 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492651b-e8f6-430a-936f-3fa6cc4ff064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:59.753492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492651b-e8f6-430a-936f-3fa6cc4ff064 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:59.765331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1435/2876] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:53:59.769761 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447150ca-673b-43d3-a879-dc16945210a3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:59.780121 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447150ca-673b-43d3-a879-dc16945210a3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:59.785122 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2b42c-2353-414a-b1e8-168e3bb5b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:59.785809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2b42c-2353-414a-b1e8-168e3bb5b3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.786099 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2b42c-2353-414a-b1e8-168e3bb5b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.786384 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2b42c-2353-414a-b1e8-168e3bb5b3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.787145 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2b42c-2353-414a-b1e8-168e3bb5b3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:59.788859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2b42c-2353-414a-b1e8-168e3bb5b3b7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:59.799421 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1442/2877] 10.222.0.47 () {58 vars in 1121 bytes} [Wed Jul 13 20:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:53:59.804125 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492651b-e8f6-430a-936f-3fa6cc4ff064 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:59.814211 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492651b-e8f6-430a-936f-3fa6cc4ff064 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:59.819250 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811570d-3c3a-4105-8514-14332269ecea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:59.820190 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811570d-3c3a-4105-8514-14332269ecea admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.820475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811570d-3c3a-4105-8514-14332269ecea admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.820784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811570d-3c3a-4105-8514-14332269ecea admin admin] PATH_INFO: `/v3/projects/19c914fb1ed248c79c824c558e80b0c7/users/b5305c5c48b1414d9c844bb07e06293d/roles/0428a514a20043c680400ebf6498c74e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.836678 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811570d-3c3a-4105-8514-14332269ecea admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c914fb1ed248c79c824c558e80b0c7, user_id=b5305c5c48b1414d9c844bb07e06293d, role_id=0428a514a20043c680400ebf6498c74e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:59.839601 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811570d-3c3a-4105-8514-14332269ecea admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:59.854707 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1436/2878] 10.222.0.47 () {60 vars in 1482 bytes} [Wed Jul 13 20:53:59 2022] PUT /identity/v3/projects/19c914fb1ed248c79c824c558e80b0c7/users/b5305c5c48b1414d9c844bb07e06293d/roles/0428a514a20043c680400ebf6498c74e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:59.920357 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51040c18-4d98-4cc7-ab9e-b8e09e78a4d9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:59.920603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51040c18-4d98-4cc7-ab9e-b8e09e78a4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:59.920904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51040c18-4d98-4cc7-ab9e-b8e09e78a4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:59.948222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-51040c18-4d98-4cc7-ab9e-b8e09e78a4d9 None None] MFA Rules not processed for user `b5305c5c48b1414d9c844bb07e06293d`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:59.949434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-51040c18-4d98-4cc7-ab9e-b8e09e78a4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:54:00.019053 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:00.025429 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1443/2879] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:54:00.037397 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811570d-3c3a-4105-8514-14332269ecea admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:00.040293 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7811570d-3c3a-4105-8514-14332269ecea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:54:00.123901 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811570d-3c3a-4105-8514-14332269ecea admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:00.128046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b756b-5bc0-4e2f-92d7-bca044ca8419 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc21faf2237d405bba03e5309d93629f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:00.128763 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b756b-5bc0-4e2f-92d7-bca044ca8419 service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:00.129048 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b756b-5bc0-4e2f-92d7-bca044ca8419 service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:00.129325 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b756b-5bc0-4e2f-92d7-bca044ca8419 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:00.138369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b756b-5bc0-4e2f-92d7-bca044ca8419 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:00.139851 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b756b-5bc0-4e2f-92d7-bca044ca8419 service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:00.148991 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1437/2880] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:54:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:54:10.027259 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:10.144944 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=66,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:35.758062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51040c18-4d98-4cc7-ab9e-b8e09e78a4d9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:35.824446 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51040c18-4d98-4cc7-ab9e-b8e09e78a4d9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:35.829563 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:35.830468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:35.830750 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:35.831029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] PATH_INFO: `/v3/users/b5305c5c48b1414d9c844bb07e06293d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:35.832291 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5305c5c48b1414d9c844bb07e06293d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:35.834298 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:35.891515 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5305c5c48b1414d9c844bb07e06293d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:35.892134 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5305c5c48b1414d9c844bb07e06293d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:35.904407 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5305c5c48b1414d9c844bb07e06293d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:35.911532 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5305c5c48b1414d9c844bb07e06293d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:35.920105 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5305c5c48b1414d9c844bb07e06293d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:35.926426 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1444/2881] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:54:35 2022] DELETE /identity/v3/users/b5305c5c48b1414d9c844bb07e06293d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:54:35.931198 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97854cbf-6d2f-42f1-8aea-790dcebd2fa9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:35.931708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97854cbf-6d2f-42f1-8aea-790dcebd2fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:35.931811 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97854cbf-6d2f-42f1-8aea-790dcebd2fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:35.950183 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-97854cbf-6d2f-42f1-8aea-790dcebd2fa9 None None] MFA Rules not processed for user `645d7df5139643969242f8464ac102fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:54:35.951504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-97854cbf-6d2f-42f1-8aea-790dcebd2fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:54:36.012651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1438/2882] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:54:36.022753 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:36.024329 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:54:36.084490 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535c00c-9209-4a67-9d52-8b1c8d44335c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:36.088165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a600953-4655-4e61-a50d-dbcaa5e2808d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:36.088916 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600953-4655-4e61-a50d-dbcaa5e2808d service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:36.089193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600953-4655-4e61-a50d-dbcaa5e2808d service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:36.089469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600953-4655-4e61-a50d-dbcaa5e2808d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:36.098408 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a600953-4655-4e61-a50d-dbcaa5e2808d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:36.099886 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a600953-4655-4e61-a50d-dbcaa5e2808d service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:36.108984 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1445/2883] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:54:36.382009 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97854cbf-6d2f-42f1-8aea-790dcebd2fa9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:36.383639 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-97854cbf-6d2f-42f1-8aea-790dcebd2fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:54:36.395933 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97854cbf-6d2f-42f1-8aea-790dcebd2fa9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:36.400977 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:36.401959 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:36.402337 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:36.402675 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] PATH_INFO: `/v3/projects/19c914fb1ed248c79c824c558e80b0c7` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:36.411146 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c914fb1ed248c79c824c558e80b0c7)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:36.413190 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:36.461734 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c914fb1ed248c79c824c558e80b0c7', 'request_id': 'req-ed0d61bf-a87e-4274-bf69-3960f7785537'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:36.468257 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c914fb1ed248c79c824c558e80b0c7', 'request_id': 'req-ed0d61bf-a87e-4274-bf69-3960f7785537'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:36.473784 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c914fb1ed248c79c824c558e80b0c7', 'request_id': 'req-ed0d61bf-a87e-4274-bf69-3960f7785537'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:36.488603 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] The token cache is being invalidate because project 19c914fb1ed248c79c824c558e80b0c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:54:36.488890 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:36.489584 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:54:36.491224 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1439/2884] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:54:36 2022] DELETE /identity/v3/projects/19c914fb1ed248c79c824c558e80b0c7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:54:46.105081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:46.485644 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:16.985909 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579a01f-733f-4c7a-a5a1-d05b0db978ff None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:16.986487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579a01f-733f-4c7a-a5a1-d05b0db978ff None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:16.986487 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579a01f-733f-4c7a-a5a1-d05b0db978ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:17.004062 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4579a01f-733f-4c7a-a5a1-d05b0db978ff None None] MFA Rules not processed for user `1578d60cec6f435ab9680b88deff76ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:17.005257 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4579a01f-733f-4c7a-a5a1-d05b0db978ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:17.063406 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1446/2885] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:55:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:55:27.055656 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=25 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:41.929882 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:41.931580 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:42.011013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d61bf-a87e-4274-bf69-3960f7785537 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:42.017956 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:42.018621 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:42.018905 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:42.019185 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:42.021651 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:42.034121 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:42.035802 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:42.045698 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1440/2886] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:55:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:55:52.041419 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=32 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:57.698529 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812c697-5b3b-46de-a338-49ae3e4c1d52 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:57.699704 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812c697-5b3b-46de-a338-49ae3e4c1d52 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:57.699988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812c697-5b3b-46de-a338-49ae3e4c1d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:57.717870 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0812c697-5b3b-46de-a338-49ae3e4c1d52 None None] MFA Rules not processed for user `4563cbf8fdbc492fae93bf92a004d0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:57.719079 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0812c697-5b3b-46de-a338-49ae3e4c1d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:57.780313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1447/2887] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:07.773681 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=25 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:24.981128 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:24.982714 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:25.001881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388ce05-1569-4387-bfb2-bce85894e31a service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:25.009359 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbeee29-8e91-4469-8874-02a51a4aba58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:25.010045 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbeee29-8e91-4469-8874-02a51a4aba58 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:25.010403 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbeee29-8e91-4469-8874-02a51a4aba58 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:25.010751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbeee29-8e91-4469-8874-02a51a4aba58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:25.012872 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2dbeee29-8e91-4469-8874-02a51a4aba58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:25.027708 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbeee29-8e91-4469-8874-02a51a4aba58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:25.029937 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbeee29-8e91-4469-8874-02a51a4aba58 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:25.043884 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1441/2888] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:25.051358 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812c697-5b3b-46de-a338-49ae3e4c1d52 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:25.053109 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0812c697-5b3b-46de-a338-49ae3e4c1d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:25.134136 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812c697-5b3b-46de-a338-49ae3e4c1d52 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:25.138611 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:25.139531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:25.139956 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:25.140236 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] PATH_INFO: `/v3/users/1578d60cec6f435ab9680b88deff76ec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:25.141631 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1578d60cec6f435ab9680b88deff76ec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:25.143822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:25.194531 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1578d60cec6f435ab9680b88deff76ec'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.195184 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1578d60cec6f435ab9680b88deff76ec'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.206923 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1578d60cec6f435ab9680b88deff76ec'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.214151 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1578d60cec6f435ab9680b88deff76ec'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.222841 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1578d60cec6f435ab9680b88deff76ec'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.229045 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1448/2889] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 20:56:25 2022] DELETE /identity/v3/users/1578d60cec6f435ab9680b88deff76ec => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:56:25.403006 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbeee29-8e91-4469-8874-02a51a4aba58 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:25.414309 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbeee29-8e91-4469-8874-02a51a4aba58 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:25.474284 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:25.475187 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:25.475492 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:25.475751 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] PATH_INFO: `/v3/projects/c26f9b9ea47148708610563386fcecaa` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:25.476969 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26f9b9ea47148708610563386fcecaa)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:25.478881 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:25.538558 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26f9b9ea47148708610563386fcecaa', 'request_id': 'req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.545489 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26f9b9ea47148708610563386fcecaa', 'request_id': 'req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.552105 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26f9b9ea47148708610563386fcecaa', 'request_id': 'req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.568451 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] The token cache is being invalidate because project c26f9b9ea47148708610563386fcecaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 20:56:25.568743 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.569495 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 20:56:25.571164 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1442/2890] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 20:56:25 2022] DELETE /identity/v3/projects/c26f9b9ea47148708610563386fcecaa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:56:35.225690 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:35.564835 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:34.759201 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:34.761440 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:34.828185 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c76f3-a3b3-4de7-a64b-3802cf6ce850 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:34.879910 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d739cb-25b2-4749-bade-de9987addf4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:34.880585 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d739cb-25b2-4749-bade-de9987addf4b service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:34.880873 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d739cb-25b2-4749-bade-de9987addf4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:34.881152 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d739cb-25b2-4749-bade-de9987addf4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:34.883212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-16d739cb-25b2-4749-bade-de9987addf4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:34.893554 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d739cb-25b2-4749-bade-de9987addf4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:34.895029 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d739cb-25b2-4749-bade-de9987addf4b service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:34.903343 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1449/2891] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:44.899901 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=46 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:12.921441 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:12.923508 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:12.986826 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bac52-aae8-40bf-a316-afe0d3a6e0a5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:12.990624 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:12.991372 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:12.991724 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:12.992012 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:12.994138 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:13.057999 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:13.059504 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:13.070689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1443/2892] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:23.064579 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=48 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:23.143657 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea9f5b-33cb-4ba2-a13e-1bd0b96fd8e1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:23.144212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea9f5b-33cb-4ba2-a13e-1bd0b96fd8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:23.144212 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea9f5b-33cb-4ba2-a13e-1bd0b96fd8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:23.162264 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-02ea9f5b-33cb-4ba2-a13e-1bd0b96fd8e1 None None] MFA Rules not processed for user `4563cbf8fdbc492fae93bf92a004d0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:23.163495 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-02ea9f5b-33cb-4ba2-a13e-1bd0b96fd8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:23.170891 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1450/2893] 10.222.0.47 () {58 vars in 957 bytes} [Wed Jul 13 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:23.209632 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:23.211260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:23.223369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be6ed7-6998-426a-8e12-cf1e3b578cda service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:23.229362 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc23c141eff42f98d31deb2d8aaaca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:23.230305 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:23.230664 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:23.230967 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:23.233873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:23.314728 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:23.316229 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:23.328355 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1444/2894] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:23.632858 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea9f5b-33cb-4ba2-a13e-1bd0b96fd8e1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:23.635081 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-02ea9f5b-33cb-4ba2-a13e-1bd0b96fd8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:23.648478 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea9f5b-33cb-4ba2-a13e-1bd0b96fd8e1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:23.653219 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3c058-daf4-4b9b-aa7f-69bfb0221ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:23.653866 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3c058-daf4-4b9b-aa7f-69bfb0221ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:23.654125 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3c058-daf4-4b9b-aa7f-69bfb0221ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:23.654434 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3c058-daf4-4b9b-aa7f-69bfb0221ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:23.656561 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-33c3c058-daf4-4b9b-aa7f-69bfb0221ea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:23.666468 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3c058-daf4-4b9b-aa7f-69bfb0221ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:23.667988 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3c058-daf4-4b9b-aa7f-69bfb0221ea8 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:23.676315 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1451/2895] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:25.344689 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:25.346192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:25.355809 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3c058-daf4-4b9b-aa7f-69bfb0221ea8 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:25.364950 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3c058-daf4-4b9b-aa7f-69bfb0221ea8 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:25.368713 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:25.369416 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:25.369697 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:25.370002 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:25.371846 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:25.385603 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:25.387075 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:25.398074 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1452/2896] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:25.422880 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550042df-779f-4b3e-97d7-bfa1f5f4e299 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:25.428331 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:25.429013 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:25.429300 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:25.429548 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:25.431760 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:25.445974 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:25.447745 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:25.458473 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1445/2897] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:35.393560 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=15 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:35.453929 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=66 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:39.017094 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:39.018741 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:39.034636 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17d72f-7d4e-44ed-9fce-0558d1d399bc service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:39.041084 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:39.041758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:39.042047 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:39.042326 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:39.044469 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:39.057395 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:39.059033 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:39.070959 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1453/2898] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:42.328294 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:42.329816 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:42.342375 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154b36f-6df1-4a89-b3db-1fc2acfbbce4 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:42.346499 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24163dff-8be4-473a-a4a9-02f7469d9c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:42.347148 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24163dff-8be4-473a-a4a9-02f7469d9c41 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:42.347437 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24163dff-8be4-473a-a4a9-02f7469d9c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:42.347754 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24163dff-8be4-473a-a4a9-02f7469d9c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:42.349668 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-24163dff-8be4-473a-a4a9-02f7469d9c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:42.421119 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24163dff-8be4-473a-a4a9-02f7469d9c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:42.422619 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24163dff-8be4-473a-a4a9-02f7469d9c41 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:42.433112 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1446/2899] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:44.081596 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:44.083193 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:44.097904 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ffae-2196-4007-a6c8-10f400e89b2e service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:44.103498 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8f1d8-0739-47ab-8a45-d7d83eaff652 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:44.104156 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8f1d8-0739-47ab-8a45-d7d83eaff652 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:44.104441 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8f1d8-0739-47ab-8a45-d7d83eaff652 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:44.104735 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8f1d8-0739-47ab-8a45-d7d83eaff652 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:44.106680 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-02a8f1d8-0739-47ab-8a45-d7d83eaff652 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:44.119990 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8f1d8-0739-47ab-8a45-d7d83eaff652 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:44.121610 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8f1d8-0739-47ab-8a45-d7d83eaff652 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:44.132774 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1454/2900] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:52.428379 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=30 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:54.128198 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=12 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:28.271477 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24163dff-8be4-473a-a4a9-02f7469d9c41 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:28.330200 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24163dff-8be4-473a-a4a9-02f7469d9c41 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:28.336965 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0d83b-11f7-46b7-a9e5-31d65604d93b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80bb5b10ce945cabbfb88ca5df92e3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:28.337699 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0d83b-11f7-46b7-a9e5-31d65604d93b service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:28.337994 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0d83b-11f7-46b7-a9e5-31d65604d93b service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:28.338753 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0d83b-11f7-46b7-a9e5-31d65604d93b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:28.340281 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f1b0d83b-11f7-46b7-a9e5-31d65604d93b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:28.374155 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0d83b-11f7-46b7-a9e5-31d65604d93b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:28.376079 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0d83b-11f7-46b7-a9e5-31d65604d93b service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:28.387235 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1447/2901] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 21:00:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:00:38.382367 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=31 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:38.425642 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8f1d8-0739-47ab-8a45-d7d83eaff652 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:38.446594 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8f1d8-0739-47ab-8a45-d7d83eaff652 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:38.450508 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf83c0d-19f6-4b8e-970c-5c521cc5fa8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c57d44c19d15496e96c6e37510ce67fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:38.451163 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf83c0d-19f6-4b8e-970c-5c521cc5fa8f service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:38.451450 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf83c0d-19f6-4b8e-970c-5c521cc5fa8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:38.451758 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf83c0d-19f6-4b8e-970c-5c521cc5fa8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:38.453637 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6cf83c0d-19f6-4b8e-970c-5c521cc5fa8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:38.552859 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf83c0d-19f6-4b8e-970c-5c521cc5fa8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:38.554351 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf83c0d-19f6-4b8e-970c-5c521cc5fa8f service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:38.574726 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1455/2902] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 21:00:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:00:41.700384 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0d83b-11f7-46b7-a9e5-31d65604d93b service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:41.723674 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0d83b-11f7-46b7-a9e5-31d65604d93b service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:41.729515 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:41.730192 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:41.730475 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:41.730750 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:41.733081 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:41.766109 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:41.768883 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:41.785596 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1448/2903] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 21:00:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:00:43.266749 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf83c0d-19f6-4b8e-970c-5c521cc5fa8f service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:43.283733 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf83c0d-19f6-4b8e-970c-5c521cc5fa8f service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:43.289252 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1f25f-7b00-4bf0-9ecf-dd0af6242211 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:43.290227 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1f25f-7b00-4bf0-9ecf-dd0af6242211 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:43.290513 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1f25f-7b00-4bf0-9ecf-dd0af6242211 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:43.290808 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1f25f-7b00-4bf0-9ecf-dd0af6242211 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:43.292832 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5be1f25f-7b00-4bf0-9ecf-dd0af6242211 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:43.308166 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1f25f-7b00-4bf0-9ecf-dd0af6242211 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:43.309966 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1f25f-7b00-4bf0-9ecf-dd0af6242211 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:43.322165 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1456/2904] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 21:00:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:00:44.733600 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:44.735029 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:44.751873 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21313651-a327-4192-8c69-276f674dc1ab service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:44.756626 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:44.757530 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:44.757815 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:44.758094 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] PATH_INFO: `/v3/users/4563cbf8fdbc492fae93bf92a004d0bc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:44.759387 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4563cbf8fdbc492fae93bf92a004d0bc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:44.761347 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:44.811947 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4563cbf8fdbc492fae93bf92a004d0bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:44.812729 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4563cbf8fdbc492fae93bf92a004d0bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:44.826260 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4563cbf8fdbc492fae93bf92a004d0bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:44.833686 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4563cbf8fdbc492fae93bf92a004d0bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:44.834602 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4563cbf8fdbc492fae93bf92a004d0bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:44.851926 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1449/2905] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 21:00:44 2022] DELETE /identity/v3/users/4563cbf8fdbc492fae93bf92a004d0bc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:00:45.040822 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1f25f-7b00-4bf0-9ecf-dd0af6242211 service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:45.062174 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1f25f-7b00-4bf0-9ecf-dd0af6242211 service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:45.121963 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:45.122765 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:45.123050 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:45.123325 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] PATH_INFO: `/v3/projects/b5387bd4427e4a4994d9847c54fbf924` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:45.124541 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5387bd4427e4a4994d9847c54fbf924)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:45.126460 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:45.168313 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:45.172189 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5387bd4427e4a4994d9847c54fbf924', 'request_id': 'req-40810449-7063-45a9-ba7e-9ceb0f923eb4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.180652 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5387bd4427e4a4994d9847c54fbf924', 'request_id': 'req-40810449-7063-45a9-ba7e-9ceb0f923eb4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.186141 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5387bd4427e4a4994d9847c54fbf924', 'request_id': 'req-40810449-7063-45a9-ba7e-9ceb0f923eb4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.200830 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] The token cache is being invalidate because project b5387bd4427e4a4994d9847c54fbf924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:00:45.201205 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.202160 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.204634 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1457/2906] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 21:00:45 2022] DELETE /identity/v3/projects/b5387bd4427e4a4994d9847c54fbf924 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:00:45.209037 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:45.210505 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:45.223766 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dd5e1-f7fa-45e0-ae3d-f2ae07b2ce1c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:45.292321 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:45.293232 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:45.293512 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:45.293785 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] PATH_INFO: `/v3/users/f58cd3f098c043848f8a057dc7f2b93c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:45.295046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58cd3f098c043848f8a057dc7f2b93c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:45.297036 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:45.349383 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:45.354769 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58cd3f098c043848f8a057dc7f2b93c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.355685 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58cd3f098c043848f8a057dc7f2b93c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.368152 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58cd3f098c043848f8a057dc7f2b93c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.374892 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58cd3f098c043848f8a057dc7f2b93c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.375564 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58cd3f098c043848f8a057dc7f2b93c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.390116 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1450/2907] 10.222.0.47 () {58 vars in 1223 bytes} [Wed Jul 13 21:00:45 2022] DELETE /identity/v3/users/f58cd3f098c043848f8a057dc7f2b93c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:00:45.571063 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:45.572682 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:45.585972 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40810449-7063-45a9-ba7e-9ceb0f923eb4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:45.657878 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645d7df5139643969242f8464ac102fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81ec70650e114b9b8d48f50c1f3f4abe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:45.658767 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:45.659057 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:45.659299 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] PATH_INFO: `/v3/projects/a7efefb29e864f08aca232bab601567d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:45.660564 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7efefb29e864f08aca232bab601567d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:45.662486 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:45.711710 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7efefb29e864f08aca232bab601567d', 'request_id': 'req-4f865531-0fb3-4ce6-927a-7efd1a88e47e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.722178 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7efefb29e864f08aca232bab601567d', 'request_id': 'req-4f865531-0fb3-4ce6-927a-7efd1a88e47e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.730222 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7efefb29e864f08aca232bab601567d', 'request_id': 'req-4f865531-0fb3-4ce6-927a-7efd1a88e47e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.742247 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] The token cache is being invalidate because project a7efefb29e864f08aca232bab601567d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:00:45.742536 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.743245 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4f865531-0fb3-4ce6-927a-7efd1a88e47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:45.744945 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1458/2908] 10.222.0.47 () {58 vars in 1232 bytes} [Wed Jul 13 21:00:45 2022] DELETE /identity/v3/projects/a7efefb29e864f08aca232bab601567d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:00:55.386433 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:55.739181 dv-cinder-849690-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:49.104369 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.106859 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:49.190638 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e1d95-db85-420e-a2f5-cae38cb03361 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.261812 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c71eba-3481-4f6b-bf27-07dfa82d080b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbafea9ed6494a3f9aeba4140a32de4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef928cdf3dcd4d7392f3749bf55bec06', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.262484 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c71eba-3481-4f6b-bf27-07dfa82d080b service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.262773 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c71eba-3481-4f6b-bf27-07dfa82d080b service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.263046 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c71eba-3481-4f6b-bf27-07dfa82d080b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.264982 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d3c71eba-3481-4f6b-bf27-07dfa82d080b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:49.276363 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c71eba-3481-4f6b-bf27-07dfa82d080b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.277958 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c71eba-3481-4f6b-bf27-07dfa82d080b service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.288083 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1451/2909] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:59.284021 dv-cinder-849690-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=58 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}